INTERNATIONAL 
ORGANIZATIONS 
AND RESEARCH 
METHODS 


AN INTRODUCTION 


Fanny Badache, Leah R. Kimber, 
and Lucile Maertens, Editors 


International Organizations and Research Methods 


Summary 


Which methods are being used and in what ways? Do we need a specific methodology 
applied to the case of international organizations (IOs)? What are the concrete method- 
ological challenges when doing research on IOs? This pioneering book answers these ques- 
tions by compiling a broad inventory of the methods developed in the study of IOs under 
the five headings of Observing, Interviewing, Documenting, Measuring, and Combining. 

Whether drawn by a specific research question or a general interest in IOs, students 
and researchers can find insights in this book to initiate and pursue a research project on 
such institutions. It provides an overview of what has been carried out so far assessing both 
well-established and more recent methods used in the social sciences and humanities. All 
contributions review a specific method (or combination of methods) in a few pages, point- 
ing out the advantages and limits as well as providing concrete tips. 

As a daily companion for academics and students alike, the book is a practical guide 
that can be used throughout the research process, from the definition of the question to 
data collection and generation, as well as during the analysis and writing phases. For bach- 
elor students, the book proposes a first dip in the wide sea of IO scholarship by providing a 
strong basis to develop their methodological awareness. For graduate and doctoral students, 
it offers an overview of the various methods available to kick-start their own research on 
IOs and gives practical cues for the development of their projects. For IO scholars, the 
book encourages them to expand the pool of methods they use to pursue relevant and 
innovative research on IOs. 

Overall it creates a space for scholars and students embedded in different academic 
traditions to reflect on methodological choices and the way they impact knowledge pro- 
duction on IOs. 
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Foreword and Acknowledgments 


Research is often—if not always—a story of informal relationships, friend- 
ship, luck, and (seizing) opportunities. This book project is no exception. 
The first seed of a book on research methods and IOs dates back to the 
spring of 2016 when Fanny Badache failed to find the right methodologi- 
cal handbook while starting her PhD dissertation. It was soon taken on as 
a collective adventure among four women (Fanny Badache, Emilie Dairon, 
Leah R. Kimber, and Lucile Maertens)—colleagues and friends—passionate 
by the research on international organizations (IOs) #1OGeeks. Scientifi- 
cally, the idea of this book results from a two-fold observation. On the one 
hand, many IO scholars have recently called for a renewal in the way IOs are 
approached, stressing the need for innovative tools to capture the intergov- 
ernmental and transnational world. On the other hand, the same scholars 
have highlighted the difficulty to conduct research on IOs, due to the need 
for “discretion” and access barriers.’ To contribute to these discussions, we 
felt compelled to bridge scientific networks (mainly divided by linguistic dif- 
ferences/barriers) and to gather scholars working on IOs but yet embedded 
in different academic contexts. 

As a result, in the fall of 2017 a call for papers for an international work- 
shop on the theme “Researching the United Nations: Rethinking Meth- 
ods of Investigation” (RUN2018) was widely distributed. We successfully 
received more than 110 abstracts leading to a two-day workshop organized 
at the University of Geneva in June 2018 with more than seventy researchers 
from various countries and academic communities. The workshop consisted 


1. See for instance the roundtable organized at ISA’s annual conference in 2019 on 
“Knowledge Without Limits? On Data, Archival Access, Copyright, and Global Commons 
in International Relations Research.” 
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of fourteen panels and two plenary sessions.” A few months later, the project 
for the present book was launched. 

This book has become the witness of ongoing discussions reflecting the 
importance of addressing research methods in IO studies. With interdis- 
ciplinarity and diversity as our leitmotiv, we gathered researchers from a 
variety of backgrounds (nationality, age, academic network, geographic and 
linguistic regions) around a common research object: international organi- 
zations that—for some of us still—embody institutions that hold the prom- 
ise of peaceful global cooperation. 
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to extend our gratitude to all participants in the RUN2018 workshop for 
recognizing the importance of discussing IO research methods and provid- 
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Dakowska, Romain Felli, Simon Hug, Sandrine Kott, Nico Krisch, San- 
dra Lavenex, Marieke Louis, Birgit Müller, Ronald Niezen, Franck Petite- 
ville, Bob Reinalda, Cécile Robert, Davide Rodogno, Tommaso Venturini, 
and Thomas Weiss. Furthermore, this conference could not have happened 
without the administrative and creative support of Blaise Dupuis, Géraldine 
Moynat, and Victor Sanchez. 

Beyond the valuable comments and feedback from several individuals on 
the overall argument of the book, such as those present in the research semi- 
nar of the Groupe de Recherche sur lAction Multilatérale (GRAM), the 
coherence and structure of this book have also benefited from the feedback 
of our students, friends, and colleagues, among which Edoardo Guaschino, 
Marylou Hamm, Michael Hutter, Emma Sofia Lunghi, Iméne Torkhani, 
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dearly thank Céline Yousefzai and Gaélane Wolff for their help in working 
on the part “75 Years of IO Studies: A Methodological Lens” and Mas- 
simo Pico for his valuable support during the finalization of the book. We 
also thank Emilie Dairon, Steffen Eckhard, and Bob Reinalda for their in- 
depth feedback on the introduction. Last but not least, we thank Elizabeth 
Demers, editor of The University of Michigan Press, for taking a chance in 


2. For the detailed program, see the event website: http://www.unige.ch/run2018/ 
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and international officials who willingly and candidly responded to 
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Introduction 


Rethinking Methods of Investigation 
and International Organizations 


Fanny Badache, Leah R. Kimber, and Lucile Maertens 


Scholars have studied international organizations (IOs) in many disciplines 
generating important theoretical developments. Yet a proper assessment and 
broad discussion of the methods used to research these organizations are 
lacking. Which methods are being used and in what ways? Do we need 
a specific methodology applied to the case of IOs? What are the concrete 
methodological challenges when doing research on IOs? This book compiles 
an inventory of the methods developed in the study of IOs under the five 
headings of Observing, Interviewing, Documenting, Measuring, and Combin- 
ing. We do not aim to reconcile diverging views on the purpose and meaning 
of IO scholarship but to create a space for scholars and students embedded 
in different academic traditions to reflect on methodological choices and the 
way they impact knowledge production on IOs. 


Why This Book? 


Methods of investigation are ubiquitously discussed in today’s social sci- 
ences. In addition to the many handbooks available on specific methods 
(i.e., interviews, survey research, discourse analysis, etc.), more systematic 
methodological discussions have recently emerged in international stud- 
ies (see for instance Aradau and Huysmans 2014; Devin 2018; Klotz and 
Prakash 2008; Schmidt-Wellenburg and Bernhard 2020; Huddleston et al. 
2022; Lamont 2022). However, the field of IO studies has yet to thoroughly 
explore these avenues. 
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So far scholars researching IOs have had access to two types of publica- 
tions. On the one hand, they can get ahold of several handbooks on IOs 
(see for instance Barkin 2013; Devin 2022; Hurd 2020; Karns, Mingst, and 
Stiles 2015; Reinalda 2013; Rittberger et al. 2019; Weiss and Wilkinson 
2018), in which methodological questions are often not put to the fore. On 
the other hand, they can learn about a set of methods in manuscripts dedi- 
cated to specific approaches to studying IOs (see for instance Miiller 2013; 
Niezen and Sapignoli 2017 for an anthropological approach). 

This book does not intend to replace these volumes but rather supple- 
ments them by gathering a wide variety of methods with a specific focus on 
their applications in IO research. By bringing together junior and senior 
scholars from different disciplines (political science, international relations, 
anthropology, sociology, history, law, etc.) and academic communities in 
Europe, North America, Asia, and Oceania, our aim is to reflect upon the 
diversity and richness of the research on IOs conducted in the social sciences 
and humanities. 

Whether drawn by a specific research question or a general interest in 
IOs, the book will help both students and researchers in kick-starting and 
pursuing a research project on such institutions. The book may be read as 
an introduction to the complex world of IOs presented in a condensed and 
accessible format. In doing so, it provides an overview of what has been car- 
ried out so far and constitutes a broad inventory of both well-established and 
more recent methods used in the social sciences. All contributions review a 
specific method (or combination of methods) in few pages, pointing out the 
advantages and limits as well as providing concrete tips. 

As a daily companion for academics and students alike, the book is a 
practical guide that can be used throughout the research process, from the 
definition of the question to data collection and generation, as well as dur- 
ing the analysis and writing phases. For bachelor students, the book pro- 
poses a first dip into the wide sea of IO scholarship by providing a strong 
basis to develop their methodological awareness. For graduate and doctoral 
students, it offers an overview of the various methods available to kick-start 
their own research on IOs and gives practical cues for the development of 
their projects. For IO scholars, the book encourages them to expand the 
pool of methods they use to pursue relevant and innovative research on IOs. 


International Organizations as Research Objects 


Instated in a founding act (treaty, charter, legal status) and embodied in a 
material framework (headquarters, funding, and staff) (Smouts 1995), IOs 
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constitute a coordination mechanism between members bringing “stability, 
durability and cohesiveness” in international relations (Duverger quoted in 
Archer 2014: 2). However, IOs are not solely a group of member states; 
they are inextricably tied to both their bureaucracies and the various (non- 
state) actors revolving around them (Weiss and Thakur 2009). IOs are also 
made up of a diversified network of individuals—from executive heads to 
staff members, as well as experts, diplomats, and civil society activists— 
constituting complex systems of interaction where multilateralism takes 
multiple forms beyond formal and institutional boundaries. We therefore 
do not essentialize IOs in this book but question them as actors, fora, and 
resources (Hurd 2020) that participate and set the stage for both intergov- 
ernmental and transnational cooperation. Such a broad definition helps 
embrace the complexity of IOs as research objects and encompasses the 
diversity of IO studies. 

Existing typologies in the literature highlight multiple types of IOs by 
distinguishing intergovernmental from nongovernmental and hybrid orga- 
nizations, with a growing interest in informal IOs (Roger 2020) and public- 
private partnerships (Andonova 2017). For instance, scholars have classified 
IOs based on membership (universal or restricted), competence or scope 
(general purpose or issue-specific), function (program or operational organi- 
zation), authority (strongly or loosely binding, strong or weak in implemen- 
tation), delegation (intergovernmental or supranational), and centralization 
(decentralized or centralized organization) (Rittberger and Zang] 2006: 10— 
12; Rittberger et al. 2019: 5-7). Although this introductory chapter does 
not adopt one particular typology to classify the IOs mentioned in the book, 
such typologies are nonetheless useful to define and delimit one’s research 
project. They draw attention to specific characteristics helpful for case selec- 
tion, comparison, contextualizing research findings, and discussing poten- 
tial generalization. 

The contributions to this book approach these diverse characteristics as 
methodological challenges rather than formal lines fixing the boundaries of 
IO studies. Indeed, they are not designed in a “one size fits all” approach: 
they acknowledge when a method is better suited for a specific type of IO 
and identify the challenges of its application in different multilateral con- 
texts (formal/informal, governmental/non-governmental, etc.), addressing 
the issue of comparability between IOs. Likewise empirical illustrations 
build on fieldwork undertaken in all regions of the world and explore IOs 
with either universal or regional membership and with either thematic or 
more encompassing mandate(s). Not only does the book address this empir- 
ical diversity, it also values the different disciplinary perspectives. The con- 
tributions inform about the methods appropriate to answer a large variety of 
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research questions including on the nature of IOs, the reasons behind their 
creation, their functioning and their role in world politics to name a few. 


Methods as a Deliberate and Reflexive Stance in the Research Process 


Scholars have access to a breadth of methodological tools to research IOs 
and have relied on different approaches when it comes to methodology. 
Indeed, methodology concerns the logical structure and procedure in sci- 
entific inquiry (Sartori 1970: 1033). Put differently, it relates to “the intel- 
lectual process guiding self-conscious reflections on epistemological assump- 
tions, ontological perspective, ethical responsibilities, and method choices” 
(Ackerly, Stern, and True 2006: 6). IO scholarship accommodates a large 
variety of views on these reflections. This book focuses more precisely on 
method choices while raising issues regarding their integration in various 
methodologies. 

In this vein we define methods as both technical and performative tools: 
they help collect, generate, and analyze data; they also construct particular 
accounts of the social world (Ackerly, Stern, and True 2006; Law and Urry 
2004). In doing so, we approach methodological choices as constitutive 
of a reflexive research process: they have important consequences for the 
production and dissemination of knowledge. Drawing from Bellamy, Wil- 
liams, and Griffin, who invited peacekeeping scholars to “be theoretically 
self-conscious and ask basic questions about what [they] are looking at and 
why, and what is excluded when [they] look at something in a particular 
way” (2010: 20), we enjoin researchers to be methodologically self-conscious. 
Whatever our ontological and epistemological traditions, one should con- 
sciously reflect on the methods one chooses and how one uses them to pro- 
duce knowledge on the world and be aware as to what these methods allow 
to grasp or lead to ignore. 

‘This reflexive stance ought to be adopted throughout the research process. 
On the one hand, by reflecting on how one’s academic training and back- 
ground influences the methods one selects, one can deconstruct the prejudices 
one may have on specific methods or methodologies. On the other hand, 
situating methods in their historical and disciplinary contexts helps reconsider 
the opportunities and challenges of their application in IO studies. 

In this book, our focus on methods is therefore embedded in a broader 
thinking on the (various) goals of scholarship (making generalizations, 
understanding particular cases, etc.) and on the consequences of using par- 
ticular methods for knowledge production (Bourrier 2017). 
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IO Research Methods? 


While acknowledging ontological and epistemological divergences on 
the role of IOs in international relations, the book calls for theoretical 
and methodological dialogue. To the question “are IOs a specific research 
object when it comes to methods?” the book reveals that IOs are both banal 
research objects in the social sciences and peculiar settings in which to con- 
duct research. Overall, each method calls for a certain degree of specification 
to suit the context, adapt to the research object, and follow the research- 
ers’ increasing familiarity with their object. Hence if there are no such “IO 
research methods,” the contributions to this book show fluctuating degrees 
of methodological adjustment. While all trace the broader use of a specific 
method in the social sciences and humanities, some insist on a necessary 
adaptation not only to apply one method to IOs but to benefit the most 
from its advantages in the context of a research project on IOs. 

Researchers may have to adapt their methods to the specificities of 
the IO context, considering two intermingled dimensions: the way they 
approach IO research and the characteristics of the IO(s) under study. First, 
the book demonstrates that the methods researchers use—and the required 
adjustments—depend on their own definition and conception of IOs often 
in line with their academic background and the paradigms they subscribe 
to. For some, states are seen as the main actors in an intergovernmental 
perspective. They therefore question power politics within IOs and, in turn, 
the influence of IOs on governments. Methods used to analyze states’ voting 
patterns, for example, inspired by the study of domestic politics and foreign 
diplomacy, must be reevaluated in light of the shift from national to interna- 
tional considerations (see part 4—Measuring). For others, IOs are conceived 
as a collection of individuals, professions, and practices. This perspective 
entails opening the box and using methods that unravel internal processes 
at play, not only those of government representatives. When doing so, the 
contributions hereafter emphasize the need to adjust methods to the speci- 
ficities of the internal structures of IOs and to the characteristics of IO staff 
(see part 1— Observing and part 2—Jnterviewing). 

Second, methodological adaptations relate to the wide diversity of 
organizations under scrutiny in IO studies. Three common specificities 
of IOs should be considered when fine-tuning methods to a research 
object: (i) IOs encompass a large variety of actors, (ii) they are multilevel 
organizations, and (iii) they are embedded in a complex political envi- 
ronment. Methodological adaptations result from a detailed assessment 
of the ways these characteristics may affect the research question and 
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expected results. Concretely, this means that a researcher must map out 
the different actors involved in order to identify the most relevant types 
of interviews, the language(s) to use, and the power dynamics that will 
impact data generation (see part 1—Observing and part 2—Jnterviewing). 
It implies considering the institutional features of the IO(s) one intends 
to scrutinize in order (i) to adjust methods to its degree of formality, (ii) 
to adapt methods of data generation and analysis to the most fruitful IO 
artifacts and productions (see part 3—Documenting), and (iii) to find the 
right combination of methods (see part 5—Combining). If these adapta- 
tions might be marginal like in the case of computerized text analysis 
or prosopography (see part 2—IJnterviewing and part 4—Measuring), in 
other instances, such as participant observation or voting analysis (see 
part 1—Observing and part 4—Measuring), IO specific characteristics 
are critical features to consider to best use these methods since IO mem- 
bership, multilevel action, and bureaucratic mechanisms require particu- 
lar methodological adjustments. In other words, the book invites the 
reader to reflect on the spatial, temporal, and institutional context(s) 
in which research on IOs is conducted and its related methodological 
implications (Maertens et al. 2021). 

Overall the empirical material emanating from IO research should 
become a solid basis from which we critically approach the data we col- 
lect and use. Data one generates is both contingent and revealing of (i) 
the varying types of access one is granted and (ii) the different ways IOs 
are willing to share but also produce and disseminate their knowledge and 
information. In this vein, contributors to this book point to concrete cues 
to be mindful of the manufactured nature of data. For instance, while 
statistical data is widely employed in the social sciences, when using IO 
statistics, one should be aware of the issues of standardization required for 
the sake of comparability between states (see part 4—Measuring). Like- 
wise the documents an IO provides depict what is made available and vis- 
ible (see part 3—Documenting and part 5—Combining). Power relations 
experienced when doing fieldwork may shape the generated data, while 
informing on the power dynamics instilled in IOs (see part 1— Observing 
and part 2—Jnterviewing). A critical stance on data is therefore indispens- 
able for each step of the research process and for every type of method 
we use. Taking stock of the visible thus needs to come hand in hand with 
a thought process around the invisible. Paying serious attention to data 
generated by researchers, or provided by the IO itself, allows for a finer 
grained understanding of IOs. 
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Seventy-Five Years of IO Studies: A Methodological Lens 


To what extent have scholars discussed research methods in their studies of 
IOs? How do they present their methodology? Can we observe an evolution 
over the past seventy-five years? To answer these questions, we conducted a 
longitudinal study in which we analyzed a corpus of research articles on IOs 
from 1945, the year of the creation of the United Nations, to 2020.' The 
corpus is based on articles drawn from the seven most used and referenced 
peer-reviewed journals according to the contributors to this book. Since 
1950 and following a five-year mark, we retrieved the articles addressing IOs 
and/or multilateralism in either the title or the abstract (see table 1 below). 

‘The systematic selection led to a total of 455 articles. Figure 1 displays 
the number of publications in our sample over time while table 2 shows that 
the increased number of retrieved articles since the 1990s also results from 
the inclusion of new journals in our sample. 

We analyzed our sample along three axes: (i) we looked at whether 
authors use first- and/or second-hand empirical data or if the article is 
mainly conceptual; (ii) we assessed whether authors discuss the methods 
used to collect, generate, and analyze their empirical data by coding the pres- 
ence/absence of explicit methodological reflection in each article relying on 
the mention of “data,” “methods,” and/or “methodology’; (iii) we checked 
whether the articles have a section/subsection dedicated to the presentation 
and explanation of the methodology and methods employed. The findings 
point to general trends throughout the entire period and show significant 
evolutions over time. 


Table |. Methods in IO Studies: Surveyed Journals 


Number of Proportion in 
Journal (year of creation) articles on IOs sample 
International Organization (1947) 151 33,19 
International Studies Quarterly (1959) 59 12,97 
European Journal of International Relations (1995) 44 9,67 
Global Governance (1995) 106 23,29 
Critique internationale (1998) 22 4,84 
Review of International Organizations (2006) 44 9,67 
Global Policy (2010) 29 6,37 
TOTAL 455 100 


1. Similar endeavors were last carried out in the 1970s, see Alger (1970) and Dixon (1977). 
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Figure |. Number of publications on international organizations per year 


Table 2. Methods in IO Studies: Sample Overview per Year 


Number of Number of articles 
journals sampled on IOs 
1950 1 10 
1955 1 12 
1960 2 14 
1965 2 35 
1970 2 18 
1975 2 16 
1980 2 6 
1985 2 7 
1990 2 4 
1995 4 25 
2000 5 43 
2005 5 47 
2010 7 54 
2015 7 64 
2020 7 100 


First, we found that the large majority of publications on IOs, since 
1945, relies on empirical data for their demonstration, with 84 percent of 
articles using first- and/or second-hand data. Our results then show that 
while most articles using empirical data propose a methodological reflec- 
tion (almost 70 per cent), 30 percent of them contain a section or subsec- 
tion dedicated to the methodology and/or methods the authors employed 
to conduct their research. 
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| | Articles using empirical data Articles without empirical data 


Figure 2. Use of data in articles on international organizations over time 


Based on these three dimensions, the analysis also shows interesting evo- 
lutions over time. First in terms of the use of empirical data, over the entire 
period we notice that scholars have predominantly relied on empirical data 
in their study of IOs, as summarized in figure 2. In terms of methodologi- 
cal reflection and section, we observe that scholars have discussed research 
methods since the beginning of IO studies. However, we see a constant 
increase in this endeavor since 1995. Figure 3 presents the share of articles 
with methodological reflection and section over time. In particular, it shows 
that the increased methodological reflection is accompanied by a formaliza- 
tion in publications. Whereas articles published before 1995 barely have a 
dedicated methodological section, from this date on, the share of articles 
including such a section has more than doubled. In our view, this trend 
could reflect an increased formalization in the way research on IOs is con- 
ducted along with shifts in publication requirements. 

In sum, we observe an increasing interest in presenting the methods used 
to produce knowledge on IOs. Our sample shows that explicit mentions of 
research methods are central to publications on IOs today. However, more is 
yet to be said and researched on whether dedicated sections actually suggest 
greater critical stances on authors’ methodological practices. This book is 
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Figure 3. Evolution of the share of articles with methodological reflection and section 


designed as a contribution to deepen methodological discussions and ongo- 
ing developments in the field of IO studies. 


How to Read This Book? 


The book comprises three types of contributions in each of the five parts 
entitled Observing, Interviewing, Documenting, Measuring, and Combining. 
Chapters present one method (or set of methods) and its application in 
IO studies. Boxes highlight more specific methodological tools and tricks. 
Interludes shed light on controversial or cross-method issues at the end of 
each section as a way to discuss the current debates around a type of method 
or “family” of methods. They typically build on the section’s contributions 
and propose a more conceptual and abstract stance on the section's theme. 
While the book can be read in chronological order, its various contributions 
also have the merit to be read as standalone, valuable inputs. 

How should researchers read this book? Methodological choices depend 
first and foremost on the research question to be answered. The book consti- 
tutes a roadmap to help navigate between different methodological options 
and select the most appropriate methods depending on the phenomenon 
one intends to research. Each chapter highlights the type of research ques- 
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tions one method allows to investigate and the results one may expect to 
find by using that particular method. By providing a short description and 
concrete examples of a method’s application in IO research, the chapters 
identify what the method entails, why it is relevant to study IOs, how one 
can apply that method to collect and analyze data, and what challenges one 
may face in applying that method. The chapters combine concrete examples 
based on the contributors’ own research experience as well as illustrations 
found in the literature. The contributions also include a selection of addi- 
tional readings to continue the exploration of each method. 

While researchers may be leaning toward specific methods (for whatever 
reasons), the book helps validate the a priori choices related to those meth- 
ods and suggests other (complementary) methodological avenues to answer 
the research question. Practically speaking, you may open the book to be 
introduced to a method you already know and may want to use, but you will 
also find key inputs and references to start a reflection on what else is pos- 
sible since the book offers an introduction to a wide variety of methods used 
to research IOs. Other options may thus appear as more relevant to reach 
your research objectives (especially since research questions and methodolo- 
gies go hand in hand). By touching on epistemological debates, this book 
can also help (re)formulate a relevant, feasible, and fertile research question. 

To facilitate the navigation among the numerous methodological 
options IO scholars have at hand, the book is structured in five main parts 
that group a series of methods around similar methodological traditions and 
types of empirical material. The first one, Observing, includes methods rely- 
ing on observation and addresses common challenges around access and 
confidentiality. The second part, Interviewing, gathers techniques to collect 
data through individuals with a specific emphasis on the useful adaptation 
for the specificities of the varying types of individuals involved with IOs. 
The third part, entitled Documenting, presents methods used to research IO 
productions, from their collection to their analysis. The fourth part, Measur- 
ing, addresses methods relying on quantitative data and the challenges to 
gather relevant, reliable, and comparable data. Finally, the last part, Combin- 
ing, provides examples of studies in which two or more methods are jointly 
used and the advantages that result from such a combination to study IOs. 

Interludes conclude each part by reflecting on the use of these methods in 
IO studies. They emphasize the debates and potential controversies around 
the selection of methods, their (not always necessary) adaptation to the field 
of IO research, and their constantly evolving uses. They show what bor- 
rowing and adapting methods from different academic fields can do to IO 
studies, as much as how the field of IO research revives and updates method- 
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ological tools in the social sciences and humanities. Overall, the book sheds 
light on the variety and vividness of research on IOs, in terms of topics, 
methodologies, and academic traditions, and advocates for methodological 
openness and diversity to foster innovations in IO studies. 


Limits and Avenues 


The book intends to start an in-depth and inclusive conversation about 
methods in IO studies. Yet our suggested format with its condensed con- 
tributions and entry through methods has its quirks. For instance, structur- 
ing another book around diverging IO characteristics and different types of 
actors could bring additional insights to the methodological considerations 
raised in this one. While we have worked toward an inclusive book, integrat- 
ing both a gender and a senior/junior balance, we must also acknowledge the 
underrepresentation of scholars from the Global South. 

Despite our effort to address a wide range of methods and methodologies 
embedded in various epistemologies and theoretical stances, the book does 
not cover it all. There undoubtedly remain gaps and silences. We identify 
four main topics that the book does not touch upon sufficiently, mostly 
because of ongoing developments. Firstly methods revolving around the 
increased use of digital devices must receive greater attention in the future. 
We recognize the impact of social media, like Twitter, and the use of online 
platforms for research, but have not given an exhaustive account of the ways 
in which social media and Internet-based research can be fully beneficial 
to carry out politically salient and academically timely research on IOs. 
‘These platforms may inform about the ways IOs communicate, disseminate 
knowledge, and in turn interact with their so-called beneficiaries as well as 
with individuals around the world (see for instance Bjola and Zaiotti 2020). 
Analyzing how IO actors use different types of software, applications, and 
digital platforms could point toward new practices of coordination, coop- 
eration, and collaboration. 

Secondly and partially connected to the first point, the book opens the 
discussion on issues of research “at a distance,” which deserves further atten- 
tion in upcoming investigations (Bourrier and Kimber 2022). In particular, 
researchers may be put at a distance deliberately by IO actors or as a result 
of specific circumstances. This became ever more relevant in the wake of 
the COVID-19 pandemic that propelled most scholars to think of their 
fieldwork differently. The impact of the pandemic on both multilateral- 
ism and academia under “home office” poses new challenges and questions 
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(Maertens et al. 2021), calling for methodological innovations (on online 
research methods see for instance Fielding, Raymond, and Grant 2008). 

Thirdly the book does not discuss behavioral methods, i.e., methods 
to empirically study “preferences, beliefs and decision-making” (Hafner- 
Burton et al. 2017: S2) that affect the choices of IO actors. They include 
various types of experimental methods such as survey-based and field experi- 
ments (for a review see Mintz, Yang, and McDermott 2011). These emerg- 
ing methods may be useful to test causal relations and provide new hypoth- 
eses in IO research. 

Fourthly the book only partially tackles the legal and ethical issues associ- 
ated with doing research on IOs. We see four avenues for future discussions 
to take stock of old and new dilemmas in that matter. First, the generation 
and use of data on individuals revolving in and around IOs raise important 
ethical questions about the protection of personal data. While this is not 
new, the use of commercial clouds as storage systems and the commitment 
to open science bring additional elements to consider. Second, scholars are 
also increasingly exploiting online content to explore various dimensions of 
IO action; such use requires ethical consideration, especially when it comes 
to content made available online by individuals. Third, the legal and insti- 
tutional obstacles to conduct research on and in IOs can be daunting. IO 
researchers should reflect on what the judicialization of science and access to 
public administrations is doing to IO studies and how to continue creating 
relevant knowledge on multilateralism regardless. Finally, the high connec- 
tion between IOs and academia on the one hand, and the growing public 
attention given to research on the other invite us to think more critically 
about the normative effects of IO studies. 

The book nonetheless introduces a timely discussion over the methods 
used to investigate IOs. We see it as a living document, which needs to be 
updated regularly. In this way we aim to inspire and encourage researchers to 
continue applying new methods and innovate methodologically, as there is a 
need to respond to ever-growing and ever-varying research questions raised 
by the constant evolution of IOs. 
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PART 1 


Observing 


Introduction 
Observing 


Fanny Badache, Leah R. Kimber, and Lucile Maertens 


Observing international organizations (IOs) allows us to grasp the machinery 
of world politics by providing a better understanding of how these organiza- 
tions work, their everyday enactment, and the power dynamics that structure 
relationships inside and outside the organization. To do so scholars hold a set 
of useful and well-established methods throughout the social sciences. 

Historically, social scientists have developed methods based on observa- 
tion to align data collection and data analysis by generating the fieldnotes 
that feed the analysis. Concretely speaking, the process of observation entails 
undergoing “thick description” to envisage all hypotheses, without foresee- 
ing, and leaving all interpretations open for future (in)validation. Obser- 
vation often calls for a certain degree of immersion leading researchers to 
experience—with varying intensity—the everyday of the observed. This 
section, entitled Observing, explores the various ways scholars can observe, 
sense, experience IOs, their advantages and challenges as well as the different 
temporalities at play. 

Observing IOs helps investigate several questions. First, researchers can 
interrogate power dynamics among different actors. For instance, sitting 
with a civil society delegation during international negotiations is insight- 
ful to capture dynamics of inclusion and exclusion within IOs. Second, 
observing methods permit to answer research questions on the culture of 
an IO, by feeling at a bodily level the practices, routines, and experiences of 
individuals involved in IOs’ work. Researchers can grasp IO daily activities 
from the very mundane ones that are often not considered of interest, even 
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left behind in interviews, to the technical ones that embody professional 
practices, or to the most strategic ones potentially hidden from public view. 
Third, observing IOs allows us to question the meaning actors give to their 
missions. Studies carried out with observing techniques share a commitment 
to understanding the reality such as it is experienced by the actors. Obser- 
vation contributes to uncover and explicate how people in particular work 
settings understand, account for, and manage their day-to-day situation. 

This section includes contributions introducing the different ways 
researchers can observe an IO along three axes. First, contributors present 
the varying degrees of immersion implied by different observing methods 
(see chapter 1—Direct Observation and chapter 2—Participant Observation). 
Second, chapters and boxes unpack the entire observation process—before, 
during, and after. On the one hand, they explore the impact of space and 
material constraints, namely the architecture of a building, the table arrange- 
ments in negotiation rooms, the different devices positioned in halls, on 
both the study and the observer (see box a—Observing spatial practices). On 
the other hand, they discuss the observers’ challenges regarding multiposi- 
tionality having to sway among various roles (see box c—Muzltipositionality) 
or having to develop an awareness regarding one’s own corporality in field- 
work, be it in office spaces (see box d—Carnal sociology) or remote areas trig- 
gering new ways of perceiving the world. Contributions then address the key 
moments when researchers decide to exit their fieldwork and their consider- 
ations for the relationships they have created (see box e—Exiting fieldwork). 
Third, contributors present data collection tools while observing. If taking 
notes has been the traditional form of data collection, contributions also 
account for interviewing techniques specific to fieldwork contexts (chapter 
3— Ethnographic Interviews) and look at how pictures taken during observa- 
tions can feed researchers’ analyses (see box f—Reversed photo-elicitation). 

While new modes of observing, such as digital ethnography (see box 
b—Digital observation) and the collaborative observations of international 
summits are increasingly being employed, the contributions give rich insight 
to understand the advantages of observing (in) an IO as well as the various 
challenges (Interlude I—Frictions of distance and proximity: Observing IOs 
in action). 


CHAPTER 1 


Direct Observation 


Kari De Pryck and Svenja Rauch! 


Anthropologists suggest examining written material issued by or negoti- 
ated within international organizations (IOs) as “assemblages of discourses 
and practices, following their trajectories and histories” (Müller 2013: 8). 
Observation of multilateral negotiation processes, international conferences, 
and summits reveals how international agreements are being crafted and 
allows exploring the complex settings in which these processes take place. 
The method thus provides unique insights into the dynamics of multilateral- 
ism, generating a wealth of research data. While scholars tend to distinguish 
between direct (passive) and participant (active) observation, we focus in 
this chapter on the former (see chapter 2—Participant Observation). 


What? 


Participant and direct observation share many advantages and challenges, 
yet there is a fine line between the two methods: direct observers closely 
watch the plot unfolding from their seats in the audience or “behind the 
screen,” whereas participant observers actively contribute to shaping the 
process. Direct observers may also use techniques such as “shadowing” by 
following the everyday life of actors (Czarniawska 2007). While participant 
observers have specific tasks to perform and responsibilities to assume, direct 
observers do not attempt to influence the international processes that they 
are studying and focus primarily on their research interests. 


1. The views expressed herein are those of the author(s) and do not necessarily reflect the 
views of the United Nations. 
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Various authors highlight the contribution of ethnographic methods 
to the study of IOs. Focusing on the World Conservation Congress, 
MacDonald, for example, explores the role of meetings in negotiating 
organizational order. Kamau, Chasek, and O’Connor (2018) also high- 
light the “cast of characters” of and the “use of process and drama” in 
international negotiations. Similarly, Miiller’s fieldwork at the United 
Nations Food and Agricultural Organization (2011) examines the role 
of the organization in the construction of its discourse on agricultural 
biotechnology. The direct observation of international deliberations is 
thus critical “to witness meaning as it is being made, challenged, trans- 
formed and translated” (2010: 259). These studies share the ambition 
to reveal the practices, interactions, relationships, and networks thereof 
that underpin the work of IOs as well as the reports, decisions, agree- 
ments, and ultimately the norms that they produce. Anthropologists 
have “explored the headquarters of [international] organizations, ana- 
lyzed the mechanisms of governance in daily practice, and followed the 
construction of an institutional identity in images and language” (Miiller 
2013: 4). Observing the “culture of negotiation” and consensus forma- 
tion within IOs, with each of them constituting a “microcosmos” in its 
own right, allows us to reveal the interests and power dynamics that 
underpin these organizations (Abélés 2011: 20). 

Direct observation, first, relates to settings (see box a—Observing spa- 
tial practices), i.e., the physical environment and the complex organizational 
culture in which international meetings take place. Different temporal and 
spatial dynamics can be observed, e.g., at the headquarters of IOs and in 
their regional or subregional hubs, or at international conferences that are 
hosted in different countries (Cragg and Mahony 2014; Worrall 2021; Ver- 
lin 2021). Second, direct observation allows identifying and tracking the 
main actors in the negotiation processes as they unfold. Different key players 
may determine the outcome of the negotiations at different stages, thereby 
gradually shaping their outcome. Weiss et al. (2009: 1) point to the interac- 
tions of three main groups of actors within the United Nations: member 
state representatives, international civil servants, and the “third UN” encom- 
passing civil society organizations, academia, and other nongovernmental 
stakeholder groups. Third, and most importantly, direct observation gener- 
ates data on the practices within an organization that are shaped through 
the interactions between different actors and their interests, and which are 
embedded in both the physical and cultural settings of the meeting venues 
as well as the overall geopolitical context. 
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Why? 


Müller highlights that organizational anthropologists “most of the time, [. . .] 
did not find their most precious materials in the official transcripts of sessions. 
They gleaned them in chance encounters [. . .], spontaneous interactions 
and by careful observation” (2013: 4). Direct observation thus allows closely 
watching how international processes unfold, examining the “give and take” 
of negotiations, rather than merely analyzing their outcomes. Complement- 
ing text-based analysis, direct observation can make “written documents 
speak by following the production of highly formalized and often opaque 
texts” (Miiller 2013: 4). Such dynamics can hardly be accessed remotely and 
include observations on how negotiators and bureaucrats “work” the room, 
how actors interact both in different types of sessions—plenaries, breakout 
sessions, coordination meetings, side events, and encounters in the hallways. 
Being a passive observer of the process instead of an active participant allows 
us to identify the dynamics that shape a given text or decision, without hav- 
ing to acknowledge one’s role in its production. 

While intuitive observations may seem trivial, they support the analyti- 
cal research work in capturing the atmosphere in the room, e.g., increasing 
political pressure during late night sessions and toward the end of the meet- 
ing. The various types of observation—during formal meetings and “float- 
ing observations” outside the negotiation room—treveal practices and rules 
within IOs and therefore prove valuable to put actors’ behavior into per- 
spective and to contextualize certain findings. Observing these “micro-social 
processes” generates valuable data on the “workings, networks, mechanisms 
of power and symbolic relations” and the “logics that guide [. . .] interac- 
tions” (Müller 2013: 4) between individuals. Focusing on people and their 
interactions, as well as on the settings and the rules in which they are embed- 
ded, provides “original bodies of knowledge” (Kubik 2009: 42) on the trans- 
actions within IOs and reveals the role of the “human” and “non-human” 
factors in multilateral entities. 

Direct observation further facilitates the identification of potential inter- 
view partners. While a list of interviewees may have been established before 
the start of the observation period, a physical presence provides the opportu- 
nity to reconsider and extend the list of actors who appear to play an active 
role in the negotiations. The findings gathered through direct observation 
may also prove critical to complement interview data and contrast the per- 
ceptions that different actors may share about a given situation. 

Lastly, direct observation allows benefiting from opportunities that may arise 
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over the course of the observation period, such as other meetings being held 
back-to-back with the conference that the researcher has set out to study and/or 
linkages with multilateral processes or debates taking place in other fora. Being 
open to “ethnographic surprises” (Bayard de Volo 2009) and ready to embrace 
them can further shape the research project as data is being collected. 


How? 


Several elements should be considered when preparing for direct observa- 
tion. First, depending on the research topic, the researcher needs to identify 
the most relevant site to observe: the assembly of an IO’s member states? Its 
secretariat? A specific program or department? A field mission? An interna- 
tional summit? This question illustrates the “multi-sitedness” (Marcus 1995) 
of the international fieldwork, in that practices are rarely restricted to single 
situations but always extend to a multiplicity of sites (geographical, tempo- 
ral, and social). 

Second, researchers often do not choose the fieldwork, but the fieldwork 
chooses them. Because the timing is not always right and because financial 
constraints and language barriers come into play, the choice of the fieldwork 
is often contingent on finding the right occasion. Access can be obtained 
through personal contact with an insider and/or through an official request. 
It can be unlimited or conditioned. Conditions may take various forms: 
access can be limited in time and space (e.g., bound to specific meetings 
and rooms) and the data collection and communication can be subject to 
internal review and approval. While individual research (e.g., pursued in the 
context of a doctoral project) may slip through the net, collective projects 
are often bound to the signature of a Memorandum of Understanding. One 
should thus keep in mind that the process of getting access is time consum- 
ing and should be anticipated. 

Third, new questions arise during fieldwork because IOs and summits 
are “sprawling” phenomena. Various meetings and conferences generally 
take place at the same time, forcing the researcher to select and prioritize 
certain venues over others (sometimes based on gut feeling). For instance, 
when attending an international summit, negotiations take place in plena- 
ries and parallel sessions and sometimes even in the corridors and during 
lunches. Side events have also become increasingly interesting moments of 
deliberation. It thus often takes time to get accustomed to the configura- 
tion of international institutions and to identify the most relevant sites of 
observation. 
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Fourth, the process of data collection/generation also requires reflec- 
tion. Researchers generally use note-taking devices (e.g., a notebook or a 
computer) to “write as much of what transpires as possible” (Fine 2015: 
533). Direct observation is a learning process through which the data 
intuitively reveals the most important (and sometimes unexpected) issues. 
Researchers may want to take notes and pictures of the material arrange- 
ments of the conference, the procedures guiding the unfolding of the event, 
the motivations that lead actors to agree or disagree, and the comments 
gleaned during breaks and the researcher’s own impressions of the process. 
While it can be relevant to write down verbatims, the speed through which 
deliberations generally occur makes it difficult to be exhaustive. When 
discussions extend through the night, the concentration of the researcher 
also rapidly decreases, and breaks are required. Direct observation thus 
remains an experience of learning by doing, which shapes the questions 
that researchers seek to answer. Reflecting on the researcher’s own role in 
IOs is critical in this regard. 


What Challenges? 


A few challenges have already been mentioned: the choice of the site, the 
conditions of access, and the selection of the important issues to record. 
We can add another three, pertaining to the analysis of the data gathered 
through observation. 

First, because direct observation produces data about activities that usu- 
ally take place behind closed doors (and sometimes even in secret settings), 
researchers must deal carefully with data collection and disclosure. In this 
context, IOs and universities increasingly ask researchers to provide infor- 
mation on data protection, to ensure that the identity of the individuals that 
are included in their studies will not be displayed without their consent. 
When investigating multilateral negotiations, researchers must also carefully 
treat any information that could impede on the process. A technique that 
is commonly used to circumvent these challenges is to complement find- 
ings from direct observation with other data, including interviews, personal 
accounts (such as biographies and books), official documents, and meeting 
records and webcasts (see part 2—J/nterviewing, and chapter 20—ZJnterviews 
and Observations)? When working on global environmental negotiations, 
for instance, a particularly relevant source of information are the reports 
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of the Earth Negotiations Bulletin, produced by the International Institute 
for Sustainable Development (IISD).° IISD releases daily or weekly sum- 
maries of major multilateral negotiations on the environment and on sus- 
tainable development, providing relatively neutral records on the ongoing 
deliberations. 

Second, direct observation is often criticized for being biased by the 
researcher’s subjectivities and for providing an incomplete picture due to 
the wide range of factors to examine. While these critiques should not be a 
reason to discard this method—all kinds of knowledge are “situated” —the 
robustness of the findings can be increased by either complementing direct 
observation with other methods or by conducting collaborative event eth- 
nography (CEE). CEE is an increasingly popular method that involves 
a group of researchers in the ethnographic coverage of an international 
event. By working collaboratively as a group, researchers can “better cover 
the multiple sites at, and thus make better sense of, a meeting” (Brosius 
and Campbell 2010: 247). Concretely, this means collaborating to cover 
events, share observations, and think through emerging questions and 
conclusions. CEE has for instance been used to cover “transnational mega- 
events” such as the Fourth World Conservation Congress in Barcelona in 
2008 (Brosius and Campbell 2010) or the Twenty-First Conference of the 
Parties to the Convention on Climate Change in Paris in 2015 (Aykut et 
al. 2017). 

Third, a last challenge pertains to the personal relationships that may 
develop between researchers and actors in the field. While proximity can 
be informative (one often learns more by listening to actors informally 
commenting about their work), it can also affect the research process. For 
instance, Bourrier (2017), in her work on the World Health Organization's 
response to the H1N1 and Ebola crises, recognized that she underestimated 
interviewees’ emphasis on intraorganizational struggles to protect them and 
safeguard her access to the organization. 


To Go Further 


Arborio, Anne-Marie, and Pierre Fournier. 2010. Lenquéte et ses méthodes— 
Lobservation directe [Inquiry and its methods—Direct Observation]. Paris: Armand 
Colin. 


3. https://enb.iisd.org/ (accessed July 31, 2020). 
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vation and Development Trade-offs at the Fourth World Conservation Congress.” 
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BOX A 


Observing Spatial Practices 


Fiona McConnell 


Focusing on spatial dimensions of international organizations (IOs)—from 
the architecture of buildings and design of meeting rooms to how bodies 
move around and interact in these spaces—brings to the fore assumptions 
about rules of behavior and sheds light on the nature of power relations 
between actors and how modes of politics are articulated in IOs. Not only 
does the design of IO spaces reflect expected modes of decorum that should 
be adhered to, but with many IOs essentially being members clubs, the 
demarcation of actors as inside or outside IOs has implications for access to 
space and how particular individuals behave within these spaces. 

Following Massey’s (2005) insistence that space is characterized by coex- 
istence and multiplicity, I have been interested in the messiness and liveli- 
ness of politics in the UN’s conference rooms in Geneva and New York. For 
inspiration in how to approach the UN as a “geopolitical site,” I turn to 
literature in anthropology, geography, and sociology that has ethnographi- 
cally examined the role of ritual performances and (in)visibility in political 
institutions (e.g., Rai 2010; Jeffrey and Jakala 2014). I have brought this 
work into dialogue with cultural geography scholarship on affective atmo- 
spheres (e.g., Closs Stephens 2016) to examine how the material and affec- 
tive setting of the UN influences behavior by and interactions between state 
and nonstate actors. 

‘Transcripts of speeches delivered at UN forums provide rather sanitized 
versions of these events. As ethnographers have argued, it is only by being in 
the room that you get a sense of interpersonal dynamics, shifts in atmosphere, 
and the rhythm of proceedings. I complemented ethnographic techniques 
of noting the comportment and interactions of participants with “thick 
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description” that was attuned to how the architecture of the space (including 
aesthetics that create a feeling of overwhelming grandeur and the allocation 
of seating) influenced both the establishment of and reception to proto- 
cols, and shifts in atmosphere when diplomatic decorum was disrupted. I 
also engaged in spatiotemporal mapping (Nippert-Eng 2015) that involved 
producing sketches of where individuals—state diplomats, UN secretariat 
staff, and representatives of minority and indigenous communities—sat in 
the room, how they moved about the space at different times during these 
multiday events, and how patterns of interactions varied in different spaces 
of the UN building. By turning attention to how bodies move and interact 
in specific spaces, and how spatial dynamics influence affective atmospheres, 
emotional registers, and auditory rhythms of particular IO events, this pro- 
vides a useful interpretive guide to approaching this “field.” 
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BOX B 


Digital Observation 


Mélanie Albaret 


Although the issue of the digitalization of diplomacy is not new, the pan- 
demic has led to a massive virtualization of multilateral debates. Scholars 
who used to observe IOs in situ have had to observe digitally (Eggeling and 
Adler-Niessen 2021; Vadrot et al. 2021). 

While IO work usually takes place in both a spatial and political loca- 
tion, spatial unity disappears in online sessions. The actors no longer con- 
verge to the same physical location. Scattered across different places, they 
meet in virtual spaces, clicking on online platform links to participate in 
meetings and negotiations. The unity of the observation field is no longer 
ensured by spatial unity (nor is it by the unity of time since the actors may 
have recorded their statements in advance in some cases) but rather by com- 
mon practices. 

Digital observation can take different forms as highlighted by the fol- 
lowing two examples. It can consist of observing IO online sessions from 
one’s own office or home while the actors under study experience that same 
distance. Scholars and actors are in some way in codistance (Albaret 2022). 
This experience differs from another kind of digital observation in remote 
surveys where only the researcher is at a distance (Bouju 2015), for example 
when scholars follow the UN bodies official sessions via UNWeb TV. 

‘The possibility of digital observation of IO work has several advantages 
beside the low financial cost. Scholars can continue the field survey and keep 
tracking the issues they are interested in even in the event of health or travel 
restrictions. Experiencing codistance, the researcher shares the daily experi- 
ence of their respondents, at least in part. Scholars can witness in real time, 
along with the actors in the field, emerging practices due to the shift toward 
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digitalization. However, these developments again raise fundamental ques- 
tions regarding observations in IOs (see chapter 1—Direct Observation, and 
chapter 2—Participant Observation). Getting access to the digital platforms 
to participate in live online sessions often requires accreditation among other 
steps (such as sending emails to get the link to join the meeting), which can 
be time-consuming. The online presentation of self and one’s own staging 
in front of the video must be carefully prepared. Scholars can also face daily 
updated ethical challenges because of the intrusion of private life in multilat- 
eral public meetings due to “work from home.” The transition from in situ 
to digital observation involves a whole set of adjustments to the researcher's 
position and furthers the need for reflexivity. 

Digital observation thus has several limitations. Some practices are no 
longer observable (especially corridor diplomacy or behind-the-scenes nego- 
tiations) or are invisibilized (reactions of people when the videos are cut, pri- 
vate discussions by chat that replace the low-voice discussions in the room 
for instance). Access to actors for informal discussions or interviews is made 
far more complicated. The distance and observing from one’s familiar place 
(home, office) hinders the impregnation process that is essential to decrypt 
IO practices. 

Digital observation can contribute to IO analysis in highly constrained 
situations provided it is firmly anchored in other data generation methods. It 
can also be deliberately chosen as an appropriate method to study issues such 
as the digitalization of multilateral diplomacy (Fielding et al. 2017; see box 
o—Artifact Analysis): such a topic may imply observing digital exchanges 
(within WhatsApp group for instance). 
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CHAPTER 2 


Participant Observation 


Leah R. Kimber and Lucile Maertens 


Participant observation provides a unique insider’s view on global politics in 
the making. By enabling scholars to look behind the doors of international 
organizations (IOs), it can contribute to study how IOs produce documents, 
discourses, and artifacts; to grasp routine activities that shape decision mak- 
ing; to observe how power is exercised between IO members and different 
organizations; and to understand how IO staff navigate between daily con- 
straints, informality, and hierarchy. 


What? 


Traditionally used in anthropology and sociology (Malinowski 1922; 
Whyte 1943), ethnography claims coherence and continuity between data 
collection and data interpretation by the same individual within a research 
endeavor insisting on “the crucial nature of manufacturing data” (Beaud and 
Weber 2010: 274). Among methods drawing on ethnography, participant 
observation is characterized by its participatory dimension, meaning that 
the researcher fulfills tasks specific to their assigned role. Concretely, it refers 
to the method through which the researcher generates data by not only 
being embedded but also actively participating in the activities under study 
while trying to embody the actors’ lives to better understand them. Immer- 
sion provides the necessary material to produce thick descriptions to situate 
action in its context (Geertz 1973). In other words, participant observation 
helps explore and depict the reality such as it is experienced by the actors. 
In political science and international relations, scholars have increasingly 
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relied on participant observation to uncover hidden power relationships and 
reconstruct actors’ strategies in accomplishing their daily activities (Bura- 
woy et al. 1991; Schatz 2009). Scholars have used participant observation 
to study IOs through a variety of entry points like embedding themselves 
as interns within IOs’ secretariats, joining a delegation during international 
negotiations, or integrating the “beneficiary communities” (Kimber 2020; 
Maertens 2016; Miiller 2013). 


Why? 


On the one hand, participant observation provides an insider’s perspective 
on the organization's functioning. First, researchers can reveal power asym- 
metries and configurations of multiple actors in constant renegotiation: they 
deconstruct a restrictive conception of an IO as a unitary actor by uncover- 
ing the diversity and complexity of actors’ behaviors, strategies, and per- 
ceptions. For example, Bellier (2013) and Kimber (2023), who were both 
embedded as members of civil society delegations in charge of drafting pro- 
posals and tasks of coordination, enact this deconstruction: Bellier unravels 
the progressive recognition of collective actors in the production of interna- 
tional norms, while Kimber investigates inclusion and exclusion dynamics 
of civil society in intergovernmental negotiations toward the ratification of 
an international framework (2023). Likewise, hired as a research assistant at 
the United Nations (UN), Maertens captures the effects of the organization’s 
hierarchy on its programs trapped between institutional collaboration and 
inevitable competition (2016). Fieldwork more broadly helps unpack the 
role of key individuals and relations of interdependence. Scholars may, for 
instance, investigate daily face-to-face interactions with what Nair describes 
as the “hanging out” technique, which proved useful in his study on the 
Association of Southeast Asian Nations (ASEAN) (2021). 

Second, participant observation provides a unique understanding of 
everyday practices, bureaucratic constraints, and organizational cultures by 
requiring researchers to perform tasks and assume responsibilities through 
which they experience IOs on a daily basis and at a bodily level (see box 
d—Carnal Sociology). For instance, embedded as an international civil ser- 
vant within the UN Refugee Agency (UNHCR), Fresia decrypts the making 
of a common professional identity (2013). Researchers can also recount spe- 
cific practices such as actors’ negotiating techniques and network building 
(Riles 2001) while experiencing the same temporal and spatial conditions 
that constrain international negotiators and IO staff (Eckl 2021; Kimber 
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and Maertens 2021). They can depict discourse production influenced by 
routine, discontinuity, and informality while identifying mechanisms of 
knowledge production and the impacts of staff beliefs, individual and varie- 
gated engagements, and day-to-day activities on the implementation of an 
organizations mandate (Atlani-Duault 2008; Autesserre 2014). 

On the other hand, participant observation also facilitates collection of 
other types of data. First, it allows researchers to compile written sources 
through access to gray literature (internal documents in particular) and the 
intranet given only to IO staff, for instance. This way, participant observ- 
ers can potentially trace back the evolution of a speech or the influence 
of different contributions through access to draft reports and comments 
from copy editors (see chapter 10—Document Analysis: A Praxiographic 
Approach, and chapter 25—Process Tracing), to unpack the selection pro- 
cess that disappears once the report is published under one logo (Gayon 
2016). Second, during participant observation, researchers can retrieve 
artifacts (see box o—Artifact Analysis) and produce visual data: sketching 
the settings of negotiations, taking pictures (see box f—Reversed Photo- 
Elicitation) or collecting images produced by the organization under study 
(see chapter 9—Visual Methods). Third, it enables to conduct ethnographic 
interviews (see chapter 3—Ethnographic Interviews) and encourages posi- 
tive answers to interviews requests. 


How? 
Preparing Participant Observation 


First knowing “when” and “where” to conduct fieldwork is not only con- 
ditioned by the research’s ambition but contingent on the subject and the 
organization. For instance, international summits are planned months, 
even years, ahead but occur over a few days, requiring availability and 
flexibility, which may not be compatible with university schedules. On the 
contrary, bureaucratic work and organizational culture might be less sub- 
ordinate to specific calendars and rather rely on cyclical work (Eckl 2021). 
Each organization performs along its own rhythm and requires specific 
accreditation processes that the researcher needs to find out about to deter- 
mine the right moment to conduct the most productive fieldwork. For 
example, to follow international negotiations, you may have to accredit 
your university the year before the summit: the lengthy bureaucratic 
process should not be underestimated. In addition, participant observa- 
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tion within IOs may also involve travels that are all the more costly when 
immersion does not get financially reimbursed. 

Second, researchers can take on different roles and positions to access 
IO perimeters. Most often they rely on internships, volunteer work, or staff 
positions within an IO (typically within their main secretariats or country 
offices), a permanent representation (or an embassy), or a delegation (be it 
governmental or within civil society groups, etc.). At a multilateral meeting, 
for instance, access depends on the type of badge—governmental delegation, 
civil society, press, and so on. This variety of access has consequences for the 
empirical material researchers can produce through participant observation. 
For instance, integrated as a civil society member, the researcher witnesses 
advocacy strategies disconnected from the secretariat’s functioning; hired as 
an intern, they are at the heart of the secretariat’s debates, meetings, and 
productions. Although one cannot always choose the acquired position and 
degree of participation, each position generates valuable data. Researchers 
must however ensure that the data generated through the given access is 
adequate and sufficient to answer their research question (see table 3 for a 
schematic overview), otherwise having to readapt the question. Concretely, 
accreditation is often the most laborious step. Such a process can take up to 
several months. While having prior basic knowledge about the organization 
and its actors is mandatory, researchers can facilitate their access by activat- 
ing a network of contacts based on previous experiences, preliminary inter- 
views, academic networks, or personal acquaintance. Finally, the researcher 
may also need permission from their institution, through a research ethics 
committee for instance, to conduct fieldwork. 

In sum, preparing and anticipating temporal, legal, relational, spatial, 
and financial dimensions positively impact the quality of the generated data 
(Bourrier 2017; Louis, Maertens, and Saiget 2018). 


Conducting Fieldwork 


Once in the field, obtaining site maps and meeting programs helps navigate 
the spaces and provides information about the nature of the site. While 
researchers must learn to emancipate from key informers, colleagues, intern- 
ship supervisors, and administration members can also be a daily resource 
for the study thanks to their knowledge of the system as well as their net- 
works. Carrying out participant observation in an IO requires a personal 
investment not only in developing relations but also in acquiring certain 
skills, in particular linguistic ones: while a good command of English is 


Table 3. Data Generation and Participant Observation in IOs 
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Position in IO Government Civil society IO operational 
perimeters IO Secretariat delegation groups activities 
Privileged rela- - IO staff - Member state —- Civil society - National and 
tions with... - Delegates delegates members local partners 
- National - Civil society - “Beneficiaries” 
governments networks - National and 
local staff 
- Expats 
Privileged infor-  - Internal - Hierarchies - Activities of - Implementation 
mationon... functioning among member advocacy, of IO mandate 
- Organizational states lobbying and - Perception 
cultures - Relations networking of IO activi- 
- Staff practices between mem- - IOs’ inclusion ties by their 
- Relations ber states and and exclusion “beneficiaries” 
between IOs IO staff dynamics of - Relations 
- Meeting prepa- - Negotiation civil society between 
rations and strategies - Relations national and 
proceedings - Official with IO staff, international 
statement members, and staff, IO staff, 
elaborations among NGOs and local 
partners 
Challenges - Technocratic - Identification - Access denial to - Disconnect 
influence with delega- member states with IO HQ 
- Conflicting tion preventing and UN staff and interna- 
allegiance access to other spaces tional decision- 


members 


making arenas 


often indispensable, the research object can require speaking another lan- 


guage and being familiar with the organization’s jargon. 


In terms of what to observe, there is always something to pay atten- 


tion to. While the researcher orients their observations based on the ini- 
tial research question, we advise to be as exhaustive as possible by rigor- 
ously transcribing all information broadly related to the research object. 
‘The researcher may either discreetly take live notes or dedicate time slots 
to write down the observations of the day (e.g., right after working hours). 
Each observation should be transcribed with elements of context clearly dis- 
tinguishing observation from potential interpretation. The notebook (be it 
traditional notepads or smartphones)—afterwards—becomes an essential 
tool to identify overlooked facts and expected anecdotes with no analytical 
significance: observations perceived as insignificant while taking notes may 
appear more interesting due to multiple occurrences when rereading them; 
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likewise, an event considered a major turning point at the time may appear 
slightly different at a later stage. Also being open to disruptions, changes, 
and “surprises” allows the researcher to seize new phenomena as they unfold, 
potentially shifting the initial research question. 

Finally, after a first familiarization phase with the field and a second lon- 
ger observation period, the researcher needs to conclude the study. While 
considering (im) possible returns to the field, the researcher can reclaim their 
role by presenting preliminary results (see also box e—Exiting Fieldwork). 


Analyzing Data 


When analyzing data, the researcher should objectify the observations and be 
aware of the possible subjective biases and emotional factors (Beaud and Weber 
2010). When it comes to restituting the final research, the notebook can be 
used in two ways: first, excerpts can provide “thick descriptions” recounting 
the context and the actors’ whereabouts and voice; second, fieldnotes are com- 
piled to account for repeated daily routines and mundane activities through 
which international politics are negotiated and IOs are enacted. 


What Challenges? 


IOs do not easily and willingly open doors, especially if the researcher has 
traditionally had a critical stance toward their missions (Weaver 2008: 14). 
First and concretely speaking, access is often a critical obstacle. IOs are safe- 
guarded organizations, which operate away from public scrutiny in a con- 
trolled and controlling environment. Yet the relative openness/closing of 
an IO already sheds light on its functioning (Pomaréde 2020). Conducting 
fieldwork within an IO may also include physical and emotional security 
risks as well as financial and practical issues (especially for observations in 
remote places, see box z—Expeditions as a Research Method). These chal- 
lenges can be anticipated, but they often require adaptation, improvisation, 
or even revisiting the research project and key questions when the field is 
totally closed off. 

Second, participant observation raises ethical issues regarding anony- 
mization and the use of data generated in informal spaces during the study. 
For instance, how to include empirical material to which the researcher has 
gained access through sources they are not authorized to quote? This is espe- 
cially important when fieldwork takes an informal shape through “hanging 
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out” outside or at the sidelines of bureaucrats’ or diplomats’ professional 
habitats (Nair 2021). First, researchers should disclose their observation 
without needing to be specific about their research question. Second, they 
may use anonymization techniques where thick descriptions include mul- 
tiple actors, events, or practices gathered within one fictional restitution, 
or where quotes are rephrased and not attributed (especially if recording 
was used to help notetaking during events where speakers were not fully 
aware of the researcher’s practice, e.g., a plenary session during a negotia- 
tion). Another solution consists in matching the information gathered from 
other sources (see part 5—Combining). Overall, academic research does not 
aim at unveiling scoops (see box v—Challenging Secrecy), it selects sensitive 
information necessary for the demonstration while respecting ethical and 
legal rules when disclosing personal data where respondents can be identi- 
fied (e.g., deleting names might not be sufficient to prevent identifying an 
individual). 

Finally, researchers should account for their positioning and prepare their 
exit as the field may be transformed due to multipositionality and respon- 
dents’ expectations (see box c—Multipositionality, and box e—Exiting Field- 
work). Long-term observations create personal relations with the respon- 
dents who often forget about the researcher’s initial motive, especially in 
the context of “expats’ bubbles” where professional and personal lives over- 
lap and where the lines between formality and informality become blurred 
(Louis, Maertens, and Saiget 2018). Researchers can also be involved in the 
production of their own primary sources, in information sharing, and in 
producing an interpretation of events that can affect the respondents’ per- 
ceptions. Yet by being reflexive of their own presence and seeking support 
from their advisor and their institution’s ethical committee, researchers can 
subdue feelings of unease and learn from their impacts on IOs: it informs on 
the production process of discourses and artifacts while providing informa- 
tion on the organization’s openness. 


To Go Further 


Autesserre, Séverine. 2014. Peaceland: Conflict Resolution and the Everyday Politics of 
International Intervention. Cambridge: Cambridge University Press. See Appendix: 
275-88. 

Louis, Marieke, Lucile Maertens, and Marie Saiget. 2018. “The Field Study.” In 
Resources and Applied Methods in International Relations, edited by Guillaume 
Devin, 95-108. Cham: Springer International Publishing. 
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Global Organizations. Cambridge: Cambridge University Press. 
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BOX C 


Multipositionality 


Simon Tordjman 


As multilateralism expands, the original and formal division of labor between 
intergovernmental decision-making processes and bureaucracies has given 
rise to composite situations marked by blurred individual trajectories and 
institutional positions that intersect various locations and systems of interac- 
tion. Multipositionality refers to the overlap of different roles performed by 
the same individual, like being a researcher and IO staff, an expert involved in 
a governmental delegation, or a civil society activist working as an IO consul- 
tant. Building up on earlier anthropological research strategies—including 
Marcus's “multi-sited ethnography” (Marcus 1995)—multipositionality can 
ground innovative research design and methodologies to capture the mul- 
tiple belongings of actors involved with(in) international organizations and 
help unpack more global phenomena, including the construction of inter- 
national regimes, decision-making processes within IOs, and their contribu- 
tions to transnational circulation processes. 

Often combined with participant observation, multipositionality pro- 
vides opportunities to better grasp the informal dimensions of the infor- 
mants social networks and professional practices in their articulation to 
broader systems of interaction (Abeles 2011). For instance, new institu- 
tional arrangements within the UN system, such as the establishment of 
UN Women in January 2010, often tend to be analyzed as a combination 
of interstate negotiations and civil society activism, taking place at the 
headquarters level. Through a four-year-long experience as a policy analyst 
within UNIFEM/UN Women, both in New York and through assignments 
in country offices in Africa, Asia, and Central and Eastern Europe, I was in a 
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position to observe and collect data about informal social networks that were 
active both within and outside institutional arrangements. 

As it requires taking into account shifting physical locations, institutional 
belongings, and social positions of actors, research strategies based on mul- 
tipositionality may yet raise additional challenges both in terms of what to 
observe and how to translate it into an academic publication. Whether they 
be embedded in national delegations as thematic or geographical experts, 
join multilateral agencies or programs as temporary consultants, or contrib- 
ute to an NGO-led advocacy campaign, researchers become involved in the 
production of their own primary sources, in information sharing and con- 
struction of interpretations of events that can in turn affect the respondents’ 
perceptions. Reflexivity then helps take into account the positionality of the 
researcher and the situatedness of knowledge (Haraway 1988). As a means 
to prevent excessive familiarization with the language or frames used by the 
informants, particular attention can be dedicated to both similarities and 
differences in the ordinary use of words, language, and references as well as 
professional and social practices across groups and locations. 

Reversely, the ways in which professional informants respond to the 
publications and behave with the researcher may provide relevant research 
insights, especially from groups seeking additional legitimacy and acknowl- 
edgment from/in the academic sphere (Massicard 2002). 
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BOX D 


Carnal Sociology 


Julie Patarin-Jossec 


‘The ethnographer’s body and the way it can inform on social characteristics 
such as age, gender, or race impacts fieldwork. “Carnal sociology” devel- 
oped by Loic J. D. Wacquant particularly emphasizes this articulation of 
corporality and the production of ethnographic knowledges. In the 1980s, 
Wacquant’s three-year immersion in a Chicago boxing gym led him to an 
important insight: his body was not only “socially constructed” but also 
“socially constructing” in the field. Through his own boxing practice, his 
body became a mirror of the norms, values, and asymmetries that organized 
his field. The ethnographer’s body was both a research object and a method 
of investigation. 

Applied to IO research, mobilizing the concept of carnal sociology 
underlines the bodily dynamic of ethnographers’ multipositionality: to focus 
on their body can lead the ethnographer to further reflexivity regarding their 
role in the field, and the consequences of this role in fieldwork data. In 
addition, it highlights IO’ power relationships and hierarchies that have 
the ethnographer involved as researchers in the same way as our informants. 

Between February and June 2017, I realized an ethnography at a United 
Nations committee in Vienna. I started as a visiting social researcher and left 
the field as an active member of a member-state delegation. This progres- 
sive implication led to various changes. I ultimately became aware of three 
dimensions related to my body as an ethnographer that could shape data. 
First, being part of asymmetrical relationships in the field. As a temporary 
and new incoming delegate, I was not only part of a hierarchy including a 
head of delegation that would impose rules and assign me tasks that I was 
assumed to strictly follow. I was also in the precarious position of an ethnog- 
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rapher asked to justify her presence as a researcher, making more difficult 
the contestation of my duties within the delegation. Second, being assigned 
gender attributes. As the youngest and only female delegate, the head of 
delegation assigned me sexist tasks such as using my “feminine charms” and 
“flirting” with male representatives from other delegations to obtain infor- 
mation (in the informant’s words). This brings me to my third point: being 
sexualized. Informants assumed my heterosexuality, which influenced what 
they would define as my usefulness within the delegation. 

In conclusion, any fieldwork is structured by social mechanisms that 
frame daily interaction, such as gender power relationships. And if those 
mechanisms are experienced by other delegates, and reveal the organization 
of the IO, they impact the very nature of collected data. The ethnographer’s 
age, gender, and position in the field enable (or restrict) the access to per- 
sons, places, and events that entirely shape the analysis. Ethnographying an 
international organization is foremost a carnal process calling for reflexiv- 
ity and self-awareness. Carnal sociology thus engages not only to develop a 
critical stance, but it forces to turn these mechanisms into heuristic mate- 
rial and to apprehend the ethnographer’s body as a significant fabric of the 
knowledge resulting from the enquiry. 
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BOX E 


Exiting Fieldwork 


Frédéric Mérand 


From 2015 to 2019, I was an embedded observer in the cabinet of a European 
commissioner in Brussels. For a total of eight months, during stays of two or 
three weeks each trip, I crashed into meetings, interviewed the commissioner 
and his staffers, observed various interactions with the press, and listened to 
their conversations in the corridor. From the outset, I made it clear to each 
cabinet member, as well as to the commissioner, that I would be taking notes, 
but that no comments or situations that could help to identify them would be 
published without their consent. This agreement was formalized in an ethics 
protocol approved by my university. In exchange, people opened their door 
to me, gave me their time, and—I cannot silence this point—extended a 
friendship that far exceeded my expectations. Exiting such in-depth fieldwork 
proved to be an important aspect of the research project. 

When I had a first draft of my manuscript, I wrote an email to each per- 
son, appending a list of all the direct quotes I wanted to use from them. In 
a detailed message, I asked them to delete the passages they didn’t want me 
to insert or specify if they wanted to remain anonymous. I only sent them 
passages where they spoke, making sure they couldnt identify others. At 
that stage, I also made sure neither the institution nor their superiors could 
have access to the material. That first stage proved trickier than expected. 
Although I gave each individual every assurance that their confidentiality 
would be safeguarded, some of them were shocked that I had taken such 
detailed notes. They felt their privacy had been violated. I had to remind 
them that, as an ethnographer, I had to write everything down but that, of 
course, I had not shared anything with anyone but them. In retrospect, I 
should have started with a phone call. 
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Then I convened a collective reading workshop in Forêt de Soignes, 
near Brussels. I sent everyone the full manuscript that they could read in its 
entirety. This was a second opportunity to delete or slightly redact quotes 
that I attributed to them, now presented in context. At the end of the day, 
they modified very few passages, usually to ask me to remove a swear word 
or to avoid hurting someone else. There was no censorship whatsoever. For 
most of the quotes, they even agreed that I would use their real first name. 
We went over each chapter together. It was a fascinating experience during 
which my twenty “subjects” helped enriching my analyses with new ideas 
and information, correcting my mistakes, proposing their own interpreta- 
tions of certain situations, engaging in contradictory debate with each other 
about specific events. The book came out a lot stronger. 

Often, scholars do not share their research findings with practitioners 
they either interviewed or observed. Having often been guilty of this neglect, 
I wanted to make sure I would, to use Marcel Mauss's expression in The Gift, 
“give back”—through this workshop and a copy of the book—to the people 
who had given so much of their time and trust to me. 


Adapted from: Mérand, Frédéric. 2021. The Political Commissioner: A European Ethnography. 
Oxford: Oxford University Press. 


CHAPTER 3 


Ethnographic Interviews 


Leah R. Kimber and Emilie Dairon 


Ethnographic interviews, typically carried out during participant obser- 
vation, allow researchers to engage with informants, in real time, to help 
unpack the field under study. It makes observations more robust, by allow- 
ing researchers to better understand the world they are immersed in through 
a dialogue that can clarify the actors’ perceptions and practices, resources and 
constraints. This method is characterized by a situated moment of mutual 
engagement between the researcher and the respondent; it also entails a 
reflection by the researcher on their own positioning and reflexivity in the 
research endeavor. 


What? 


“Ethnographic interviews” as a composed term in the social sciences car- 
ries within itself two significant words. They both refer to a certain way of 
doing research and rest on a long disciplinary and cultural research heritage. 
On the one hand, interviews—which emerged from sociology in the United 
States (Beaud 1996)—have commonly been used to allow researchers to col- 
lect information, know-hows, and experiences linked to a particular research 
question. They rely on the relationship established between the researcher 
and their informants to produce, together, an understanding in a particular 
time and space. 

On the other hand, ethnography is a method used to generate qualitative 
data. Originally used in “terra incognita,” for the study of unknown cultures, 
its use in policy analysis is well-established since the School of Chicago in 
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the 1950s. Drawing on an anthropological tradition, this method follows an 
inductive approach that calls for the creation of theory out of data, charac- 
terized by its back-and-forth movements between data and analysis (Becker 
1958). In that vein, “the ethnographer participates, overtly or covertly, in 
people’s daily lives, for an extended period of time, watching what happens, 
listening to what is said, asking questions; in fact collecting whatever data 
are available to throw light on the issues with which he or she is concerned” 
(Flick, von Kardoff, and Steinke 2004: 225). 

Ethnographic interviews thus lie at the junction between interviewing 
and ethnography work, reflecting a moment of verbal exchanges, where 
researchers meet and talk with informants (Hockey and Forsey 2012). The 
opportunities for such interviews—mostly carried out in times of partici- 
pant observation (see chapter 2—Participant Observation)—vary according 
to the conditions in the field because they strive to be a “normal” and fluid 
interaction, to erase the artificiality of the interview situation (Olivier de 
Sardan 1995). 

Drawing on a long tradition in the ethnography of aid and agencies 
(Lewis and Mosse 2006) and renewed in international relations (Louis, 
Maertens, and Saiget 2017), especially for the study of IOs (Bourrier, 2017; 
Kimber, 2023), ethnographic interviews are a well-recognized tool among 
IO scholars despite rarely being accounted for and defined as such (Kezar 
2003). Ethnographic research often focuses on the conditions of access 
and disclosure of information during fieldwork in IOs (Dematteo 2011; 
Pomaréde 2020) but rarely discusses the process of conducting the conver- 
sations that arise from being in situ. This method, often taken for granted, 
helps reveal the context of investigation. 

Practically speaking, ethnographic interviews can take place spontane- 
ously (e.g., a random meeting at the cafeteria, an interaction at the end of 
a formal meeting) or be planned by the participants. Similarly, and in both 
cases, they can emanate from a semistructured interview guide or rather be 
questions that spark out of a specific moment, in a nonstandardized way. 
Even if the questions preexist, a researcher may take the opportunity to 
launch a discussion on a topic with specific questions, tail an interviewee’s 
flow on another topic, ultimately informing the researcher's question. 


Why? 


From the interviewer’s standpoint, the benefits are two-fold. First, the 
quality of information collected during ethnographic interviews is high, as 
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it can help identify elements that usually “resist” to more formal moments 
of the research: unexpressed values, tacit knowledge, daily practices, and 
unspoken habits. Because researchers and informants are both experienc- 
ing a situation in real time, and because the researcher instills a “natu- 
ral” and “spontaneous” conversation, a sense of mutual trust (a crucial 
aspect in elite interviews, Kezar 2003; see box j—Asymmetrical Interviews) 
emerges. That way, the interviewee is likely to disclose more information 
and to enrich the observation, e.g., by identifying other events that would 
be relevant for the research, or by suggesting that the researcher be in 
contact with other actors commonly referred to as the snowballing effect. 
In the context of IOs, at the “micro” level, it allows researchers to under- 
stand the variation in meaning among different individuals, according to 
their status (IO staff members, member state representatives, civil soci- 
ety actors), their institution (e.g., WHO, ILO, UNDP, etc.), their func- 
tions (policymakers, negotiators, administrative workers), or their sectors 
(development, humanitarian). At the “macro” level, it enables researchers 
to get a sense of the hierarchies (who works for whom under what circum- 
stances), helps clarify a process, and comprehend particular acronyms and 
a specific jargon. At the “meta” level, it helps to triangulate data, stemming 
from observations and interviews. 

Second, it allows researchers to develop a reflexive approach and to 
determine the biases they have when undertaking the study: “The reflex- 
ive researcher thinks through their own assumptions and how they affect 
the research project and shares some of these insights with the interviewed 
person” (Kezar 2003: 401). Acknowledging the biases is a first step, but 
managing them is part of the research (Olivier de Sardan 1995). Since the 
ethnographic interview is an engaged interaction, it helps researchers con- 
front their own thoughts and prejudices, thus enriching the whole analysis. 
Typically researchers can ask questions to actors in the field about a phe- 
nomenon occurring in real time. Furthermore it can help shape the overall 
research question by finding out answers to questions defined prior to the 
observation or give way to reformulate them. This practice thus avoids mis- 
interpretations and misunderstandings by the researcher. 

Ina nutshell, ethnographic interviews not only allow researchers to further 
open the box (Maertens 2016) but are also part of a larger strategy, combin- 
ing several forms of data production to validate researchers interpretation of 
social phenomena (Olivier de Sardan 1995; see part 5—Combining). On the 
one hand, while never used alone, they have value in making direct or par- 
ticipant observations more robust (see chapter 1—Direct Observation and 
chapter 2—Participant Observation). On the other hand, while not exclusive 
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to other types of interviews, ethnographic ones—during fieldwork—can 
perfectly be combined with “one-shot” semistructured interviews or bio- 
graphic interviews (see chapter 6—Biographic Interviews). To shed light on 
how international organizations think (Garsten and Jacobsson 2011), ethno- 
graphic interviews help give different in-depth cues and perspectives about 
the context under which IO actors go about their activities by exploring 
how they think and act inside the organization (Abélés 2011; Müller 2013). 


How? 


However ethnographic interviewing requires being reflexive, but also mind- 
ful and hence adaptive to the context. 

Once access is guaranteed and conditions are defined, choosing to do 
ethnographic interviews with particular individuals, working in and around 
IOs, assumes that particular individuals have valuable information to under- 
stand the field under study. Ethnographers should follow the actors in their 
different sites of action to bring themselves closer to the subjectivities and 
singularities of each informant (Müller 2013) while building on their com- 
mon experience (Spradley 1979). The choice of interviewees depends on the 
research question, but also on the windows of opportunity swaying between 
the formally planned with a specific time and place or the more spontaneous 
arising from an interaction. 

In the former, questions are predefined, in an interview guide written on 
a sheet of paper which the interviewer may or may not have in their hand. 
In the latter, questions can be “open-ended” and emerge under the principle 
of “walking and talking.” We might feel compelled to engage in a small talk 
with the person next in line at the UN cafeteria or share a random com- 
ment with someone passing—by attempting to fall naturally into the flow 
of their movement so as to find an entry point for continuing the discussion 
(Halme-Tuomisaari 2018). 

Of course such adjustments imply having to spend time in the field, to 
get to know the people (Nair 2021) (and actors to get to know the researcher) 
and learning the codes around interactions in a specific ecosystem (Dairon 
and Badache 2021), hence acknowledging the influence of time and space 
(Maertens et al. 2021). What holds true in New York, for instance, may not 
hold true in Geneva nor in field duty stations. This might mean having to 
schedule a formal interview, arranged during a negotiation break, or engag- 
ing in a conversation, in an informal manner, on the way to the cafeteria on 
a coffee break. Selecting a “good” interviewee can thus appear as a tedious 
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task, but this skill improves as the researcher’s sense and confidence increases 
throughout their fieldwork (Beaud 1996). 

Concretely speaking, carrying out ethnographic interviews requires 
availability and flexibility, as the opportunity for an interaction may arise 
at any time including at the end of a very long day, or just before another 
meeting. Heightened awareness of what has been said, the moments of a 
break between meetings, the ideal time of resuming the discussion are key to 
successful ethnographic interviews. 


What Challenges? 


We highlight three main categories of challenges, related to different stages 
in ethnographic interviewing: 


Before: Selection and Preparation 


Long immersion is key to being familiar with the field under study. Though 
it might take time for interviewees to move beyond the small talk—for they 
may become eager and more receptive once the researcher becomes famil- 
iar with the internal and external politics and is thus “fluent” enough to 
ask trenchant questions (Weaver 2008)—initial interviews and observations 
have value in identifying the key actors and the crucial moments. 


During: Interview Questions and Expected Level of Information 


Because interviewees might find it rude if a researcher relies on a written 
interview guide, they must memorize the sequence of questions or at least 
particular questions from the interview guide. Keeping up with the actors 
in their activity, by asking adequate questions for the given moment, allows 
researchers to retain their credibility in the field. 

Researchers need to continuously adopt an inquisitive mindset to handle 
events as they unfold, for this mindset will help build their confidence in 
dealing with interviewees over time. First, as ethnographers tend to become 
native themselves, they need to ensure they have various sources of empirical 
material—not solely relying on the ethnographic interviews—to triangulate 
information. The researcher thus must continuously make themselves feel as a 
stranger while at the same time integrate the field as a native. Second, consid- 
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ering that this is a jointly constructed relationship, the interviewee might also 
be tempted to give personal opinions on the research work, over the course of 
the interview, for instance. Interviewees may comment on the chosen meth- 
ods or the choice of research topics, which the researcher must be ready to 
welcome politely without becoming defensive. 


After: Transcribing and Interpreting the Material 


Transcriptions are often an overlooked methodological challenge (Beaud 
1996). As crucial information may not always be immediately transcribed 
on the spot, but rather in full, hours later, isolating ourselves in a restroom 
for example to take quick notes aside from informants can be helpful and 
even necessary. Finding tricks to remember the valuable input can go as far 
as taking quick notes and/or recording voice messages on a cell phone. 

Finally, the challenge around ethics is prominent. The crucial question 
about ethnographic interviews thus relies on asking where did this informa- 
tion come from? When was it shared? In what context? Important informa- 
tion may be easily revealed in informal settings (i.e., “walking and talking”) 
and used for research purposes without the informant explicitly being asked. 
In this case, it is up to the researcher to choose what to use in the research. 
Provided that there is no risk for the informant (anonymity) and for the 
researcher (no disclosure of confidential info that could potentially “close” 
the field for further research), it is common that researchers use information 
collected during moments when they were “not supposed to” do research. 
The “trick” of becoming close to the “natives” gives the researcher a privi- 
leged position and allows them to create ties to the point of creating friend- 
ships, but must not come at the expense of individuals’ dignity and respect 
when mobilizing the information they provide the researcher with. 


To Go Further 
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BOX F 


Reversed Photo-Elicitation 


Leah R. Kimber 


Taking pictures during fieldwork has been a common practice for social sci- 
entists in ethnography. Exploring society through photography encourages 
the researcher to “think with their eyes” (Becker 1974). Two methods have 
been traditionally at play (Meyer 2013): the use of photographs as a means 
of presenting social research where photographs are a set of data (Schwartz 
1989; see chapter 9—Visual Methods), and the use of photographs as data 
generators called photo-elicitation. The latter is a long-established method 
in visual sociology and can be used in conjunction with qualitative inter- 
viewing (Hogan 2012). Here I suggest a twist in photo-elicitation to get 
researchers to “see” more of their own fieldwork. 

Fieldwork in IOs can be tedious, for taking notes depends on the role 
we are assigned to as researchers (see chapter 2—Participant Observation). 
At UN headquarters in Geneva, I got into the habit of taking pictures with 
my smartphone to feed my notebook. They kept a visual record of the 
places I saw, revealed an atmosphere, a seating arrangement, and served as a 
memo. With no aesthetic intention, they had value in that they informed or 
reminded me of a certain context (Meyer et al. 2019). 

The process of reversed photo-elicitation started after my immersion 
when I presented the pictures to faculty colleagues. My colleagues asked me 
questions about the field under study and used my pictures to clarify their 
own queries about my field. They were relying on my data to seek additional 
information and deepen the analysis I had provided. I thus experienced a 
turnover of the situation. I was now the actor—the data generator alongside 
my pictures—and my fellow colleagues, the researchers, were triggering the 
data. According to traditional photo-elicitation, I became the actor special- 
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ized in the field who was able to provide additional information based on 
photos, hence calling this method reversed photo-elicitation. This undertak- 
ing enabled me to gain control over two processes: distancing myself from 
the field by having to explain and justify the reason I took a certain picture 
and shedding new light on my field through the input of peers. New ideas 
and theoretical approaches emerged. 

From as a simple memo that kept track of what I was unable to describe 
rapidly in writing, pictures first became an illustration to reveal the “world” 
under study and then data generators for my peers who benefited from my 
photo-elicitation skills—thanks to my situated expertise—to develop a new 
outlook on my own research, all the more relevant in an inductive approach. 
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INTERLUDE I 


Frictions of Distance and Proximity 
Observing IOs in Action 


Birgit Müller 


Anthropologists tend to observe international organizations in action, 
interested less in what they are than in what they are doing. This makes 
their approaches so diverse. When they observe the impact, the material 
and symbolic effects of international policymaking on concrete lifeworlds in 
situated places, the methods they use depend on their engagements in the 
field and pragmatically on the concrete questions that arise on-site. When 
exploring IO headquarters, anthropologists focused on the daily practices of 
governing and followed the construction of institutional identities through 
images and language. They closely observed the interactions, uncovered dis- 
sent among and between groups, and unpacked the emergence of disputes 
and the formation of consensus (see chapter 2—Participant Observation). 
Most of the time, they did not find their most precious materials in the 
official transcripts of negotiating sessions. They gleaned them in serendipi- 
tous encounters (Hertz 2010), spontaneous interactions, and through care- 
ful observation. The strength of ethnographic approaches is to take the time 
to understand, to dare deconstruct the seemingly obvious all of which by 
observing daily interactions and routines and by engaging with actors while 
keeping a distance (Müller 2013a). 


Observing lOs: A Diversity of Approaches 


From my own work on international organizations, it became evident 
that there is not just one toolbox but multiple approaches. Like any 
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social scientist who relies on ethnography, I had to constantly review my 
methods. My first research on the WTO negotiations in Seattle in 1999 
started out in the street, following and observing the different forms 
of protests, attending the side events, the staging of the counter sum- 
mit and its articulations with the official event (Miller 2000). The next 
research was completely different. It took place at the heart of the Food 
and Agriculture Organization of the United Nations’ (FAO) adminis- 
trative headquarters in Rome. From within the administration, I fol- 
lowed the way in which one of the most controversial FAO reports—the 
SOFA 2004 on “Biotechnology. Meeting the Needs of the Poor?”—was 
produced (Miiller 2011). I first started out by analyzing the report and 
then traced its social life in the institution by both interviewing admin- 
istrators and talking to members of civil society organizations, as well as 
examining the hundreds of entries contributed to an online consultation 
(see chapter 3—Ethnographic Interviews). 


The Material You Produce Depends on the Angle You Take 


The methods and consequently the material that can be collected and gener- 
ated depend on the angle taken for the research and the conditions of access 
to the field. When following a global event such as Rio+20 or COP21, inter- 
national organizations do not appear as confined entities but rather as dis- 
positifs (Foucault 1994) (assemblage or apparatus). They constantly attract 
new players, involving them as interlocutors, consultants, and experts and 
formatting them through forms of calculation, technical reasoning, and 
capacity building (Müller 2011). On the other hand, when intervening 
on the local level, international organizations represented by their experts 
appear as a single coherent actor producing normative frameworks and 
bringing into effect relationships of power and control from the metropolis 
to the remotest parts of the world. 

The complexity of international organizations becomes all the more 
apparent during major multilateral events. To make even remotely sense of 
what was going on in the multiple forums of the Rio+20 Conference in 
2012 and at the Climate Summit COP21 in Paris in 2015, we attended 
these events as a multidisciplinary team of anthropologists, historians, econ- 
omists, and sociologists and produced collective event ethnographies for 
both forums. We prepared for the event together and then explored it from 
different angles. Some of us observed the negotiations themselves, while oth- 
ers were focusing on civil society events, the indigenous peoples’ caucus, the 
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closed meetings of the business sector, and so on (Dumoulin Kervran 2021;, 
Aykut, Foyer, and Morena 2017) 

The cooperation with the other research teams allowed each team to 
go deeper in their analysis. Part of my own research, for instance, focused 
on the minute details of the negotiation of a seven-line paragraph in the 
Rio declaration. The paragraph dealt with the international governance of 
agricultural investments, an issue crucial in another international forum, 
the Committee of Food Security, where negotiations on guidelines were to 
begin in 2012. I knew the wider political context and hence could guess 
the implications of the interactions between the negotiators in the three- 
hour-long debate. The more subtle powerplays with language, intonation, 
and voice only became apparent when I analyzed the recordings back at 
my office in France together with the linguist Gilles Cloiseau (Miiller and 
Cloiseau 2015). Combining anthropological observation and linguistic 
methods enabled us to uncover the ways in which negotiators interacted 
in a highly coded language, how they “performed,” by exploring, playing 
with, and twisting the grammatical structures of the spoken language (see 
chapter 11—Discourse Analysis). 

When observing the meetings of the business sector at the Rio+20 
conference (Benabou and Miiller 2015) and at the COP21 climate nego- 
tiations in Paris (Benabou, Moussu, and Miiller 2017), again it was the 
serendipitous encounters that gave us the most interesting material for 
analysis. In Rio, for example, I found myself sitting around a table in a 
hotel ballroom with the CEOs of the world’s top four mining companies, 
openly recording our table discussion on sustainability reporting for global 
ventures. The discussion revealed their dividedness on the central issue 
of international governance; should the states be imposing constraining 
regulations on corporations or rather should the corporations themselves 
be trusted to act responsibly? 

Observing the effects of international governance on the local level 
again provided a different angle (see chapter 2— Participant Observation). I 
was conducting fieldwork in a Nicaraguan village when the Food Security 
program of the FAO was implemented there. I used material found on the 
Internet to reconstruct FAO’s interventions into the conception of food 
sovereignty laws in Nicaragua. I then followed two FAO food security 
projects at the local level over several years, observing meetings, official 
visits, talking to farmers, and accompanying them to the fields (Müller 


2013b). 
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Observational Participation in lOs: Access, Positionality, 
and Ethical Considerations 


Access to places where interactions take place and where relationships of 
trust can be established is crucial for ethnographic fieldwork as the differ- 
ent contributions to this book show. When an international organization 
is powerful, it has mechanisms to impose its standards and decisions at the 
global level and affect the interests of big businesses and capital holders. It 
then becomes difficult to gain access to them. Discretion is required. Doors 
to the negotiating rooms are closed to the external observer. Anthropologists 
are then asked to “do fieldwork without taking note” (Dematteo 2011). 
In contrast, the areas of “soft” international governance, such as the sec- 
tion of the World Intellectual Property Organization (WIPO) dealing with 
traditional knowledge (Bendix 2013), often seek to exchange with social 
scientists to circumscribe their fugitive objects and find ways and means 
to transform them into legal objects. As an expert on cultural knowledge, 
Regina Bendix was asked, for instance, to highlight how the delegates them- 
selves went through processes of acculturation inside WIPO (Bendix 2013). 

To access their field-site, social scientists take on a variety of roles, some- 
times combining research with expertise (Fresia 2013) or, on the contrary, 
disengaging in order to conduct a more detached analysis. For some research- 
ers, the difficulty consists of getting access to information in relatively closed 
organizations. For others who were able to immerse themselves, the chal- 
lenge is to keep a distance and negotiate a way out of their role as experts 
(Mosse 2008; see box c—Multipositionality). As observers, they are rarely 
disconnected from the issues at stake. Their complex position, at times close 
up, at times remote, may lead the organization to expect a certain degree of 
allegiance and discretion from the researcher (see chapter 1—Direct Obser- 
vation and chapter 2—Participant Observation). 

The most difficult and ambiguous situation I put myself in as a social 
anthropologist of IOs was the observational participation (see chapter 
2—Participant Observation) in the two-year-long negotiations on guidelines 
for responsible agricultural investment. From 2012 to 2014, I became a 
technical advisor and, in a few instances, even negotiated on behalf of the 
Civil Society Mechanism in the UN’s Committee for Food Security. Not 
only did I clearly take sides in the negotiations, but I also had access to the 
strategizing of the Civil Society Mechanism and to its internal frictions, 
which I deontologically and ethically could not share with the outside world. 
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‘The guidelines that we negotiated were only in appearance “soft” texts. As 
they defined normatively the direction that agricultural investments should 
take in the future, they challenged high economic stakes and were subject to 
fierce battle and high emotions. Practically and emotionally involved in the 
negotiations, I was unable to step back, observe, and write down not only 
my own experiences but also those of others. I accumulated mountains of 
material, lots of notes but few fieldnotes, and I have to date written little 
about this experience (McMichael and Müller 2014; Müller 2019). 


Repoliticizing International Language 


Most anthropologists who have studied international organizations do not 
define themselves as IO scholars. It is not so much the organizations as such 
that are of interest to them, but rather what happens among the people 
inside the dispositif of international governance. How are the ideas produced 
and how do they travel within international settings? (see box r—Study- 
ing Ideas) What are their impacts? Furthermore, how does the “real” world 
get translated into international language? Working with a linguist—as 
mentioned above—provides complementary theoretical tools to bring the 
international language to life (see chapter 10—Document Analysis: A Prax- 
iographic Approach and chapter 11—Discourse Analysis). Etiquettes of speak- 
ing, acronyms, and an intense haggling around terminology and word choice 
leading to heavily coded and often opaque and “technical” texts (Riles 2000; 
Merry 2006) are deconstructed. For the outcome of negotiations, language 
practices can be deeply consequential (McMichael and Miiller 2014). Each 
commitment made in a global forum slowly adds new layers of global gov- 
ernance and multilateral terminology and serves as a reference for future 
negotiations in the patchwork of international institutions, public or private 
(Biermann et al. 2009: 16). 

Ethnographies following the political processes that produce technical 
and seemingly apolitical IO reports, guidelines, and so on are instrumental 
for understanding what is at stake. In the multiple forums, nonstate actors, 
representatives of corporations and civil society organizations, use the rules 
of the game of international institutions for their own purposes and also for 
their relations to the nation-states. Furthermore, in the life of projects on the 
ground, seemingly technical issues are repoliticized (Müller 2013b; Louis 
and Maertens 2021). What is experienced as antipolitics, as the dissolving 
of conflict in a discourse of harmony, is thus less an essence than a recurring 
practice of IOs that has to be observed “at work.” 
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PART 2 


Interviewing 


Introduction 
Interviewing 


Fanny Badache, Leah R. Kimber, and Lucile Maertens 


Since the development of IO studies in political science, international rela- 
tions, and anthropology, interviews have been used as key entry points to 
the production of knowledge on IOs. Interviewing IO delegates, heads, and 
staff has allowed researchers to go beyond the written productions usually 
analyzed in law and history. In this second part, entitled Interviewing, con- 
tributions present the different types of interviews used to investigate IOs. 
Interviewing methods rely on the premise that interrogating individuals 
generate relevant scientific data. They gather different techniques flexible 
enough to be employed in various disciplines and epistemological contexts. 
Indeed, social scientists apply interviews for analytical or descriptive pur- 
poses, selecting the most useful type of interviews to answer their research 
questions. Three dimensions distinguish the different types of interviews: (i) 
the focus (an individual’s life trajectory, the description of past events, pro- 
fessional practices, perceptions, etc.); (ii) the format (survey, semistructured, 
biographic, etc.) and its support (written document, face-to-face, online, 
etc.); and (iii) the interviewees’ profiles and the subsequent impact on the 
relationship with the interviewer. The contributions give practical examples 
of different types of interviews and their applications while shedding light 
on the struggles specific to the case of IOs (see chapter 4—Surveys, chapter 
5— Semistructured Interviews, and chapter 6—Biographic Interviews). They 
discuss the tricks of the trade for each interview method that does not only 
depend on the research question but also on the types of individuals they 
encounter, be it governmental delegates, high-level bureaucrats (see box 
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j—Asymmetrical Interviews), staff members, so-called local beneficiaries (see 
box i—Jnterviewing “Beneficiaries”), civil society and private sector partners, 
and so on. 

Conducting interviews is valuable in the study of IOs, with the numer- 
ous research questions that can be answered through interviewing falling 
into three categories. First, interviewing techniques help gather informa- 
tion about past events and individuals’ life stories. For instance, they help 
recollect decision-making steps, trace back international negotiations, map 
out an organization's life course, and track the production process of an IO 
report. They also provide systematic data on the profile of interviewees when 
using surveys designed to study the socialization of IO staff (see chapter 
4—Surveys) and biographic interviews digging up individual’s professional 
path (see chapter 6—Biographic Interviews). Second, interviews generate rel- 
evant data to study perceptions. By asking IO actors about their views on 
their own daily activities, researchers can uncover the meaning that these 
actors give to their practices, their shared background knowledge, and their 
vision of the organization. Used with different individuals of the same IO, 
interviews capture critical features constitutive of an organizational culture 
while producing data on power dynamics and hierarchy as they are experi- 
enced and recollected by IO actors. Third, interviews are relevant methods 
to investigate institutional discourses, professional narratives, and more gen- 
erally how IOs present themselves to the world. By focusing on the rela- 
tionship between the interviewee and the interviewer, researchers can learn 
about professional habits and individual characters, while identifying the 
public image an IO (especially through its spokesperson or head) or a spe- 
cific department intends to display. 

This section gathers chapters presenting different types of interviews. 
Boxes stress important challenges readers should pay attention to when it 
comes to the set-up of a (collective) interview and relationships with and 
among interviewees (box h—Focus Groups) as well as the practical strug- 
gles of conducting interviews in different languages (see box g—Languages 
and Interviews) or using online platforms (see box k—Online Interviews). 
Since they involve individuals, interviews also raise important ethical ques- 
tions: researchers should reflect on their own role and potential impact as 
interviewer as much as be vigilant when analyzing and quoting interviewees 
(see Interlude II— Controversies in Interview Research). While new modes 
of interviewing, such as using voice messages, are constantly being devel- 
oped, readers can build on these contributions to select, adapt, and exploit 
interviewing methods in combination with other relevant methods for their 
research projects. 


CHAPTER 4 


Surveys 


Clara Egger and Monique J. Beerli 


Surveys are widely used in contemporary society to know, govern, and serve 
populations, from censuses and public opinion polls to clinical trials and 
consumer experience studies. They are, however, far less common when 
it comes to unpacking global governance and the critical role individuals 
play in determining IO action at different levels. With handbooks on sur- 
vey research abound, this contribution draws from ongoing research on aid 
workers to situate the interest and intricacies of doing survey-based research 
on IO professionals. 


What? 


Historically, surveys emerged as an instrument of government (Desrosiéres 
2010), first used by nascent state administrations and then social reformers 
(Converse 1987). From the 1930s onwards, surveys made their way into 
scientific practice, notably in the social and medical sciences. 

Though commonly reduced to its initial definition as a quantitative 
research tool adept in the “systematic” collection of a large quantity of infor- 
mation on a “sample” of individuals, surveys have been subject to method- 
ological pluralization. This is best evidenced in their adaptation in mixed 
methods and experimental approaches. Putting aside discrepancies in how it 
is used, survey research rests on two key premises. The first assumes that the 
characteristics, behaviors, attitudes, and past experiences of individuals can 
be analyzed through the administration of well-formulated questions (de 
Leeuw, Hox, and Dillman 2008). The second premise is that findings result- 
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ing from the study of a representative population are generalizable to the 
identified population as a whole, thereby making sampling so important. 

In IO studies, the use of surveys is fairly recent and rather rare. In addi- 
tion to the general difficulty of administering a survey, this absence may 
be explained by the fact that IR scholars rarely put the individuals who 
represent (i.e., delegates) or work for IOs (i.e., secretariat staff, consultants, 
civil society or private sector partners) center stage. As a result, we lack a 
macro-perspective on the trajectories, attitudes, interests, and practices of 
individuals who make IO action possible. 


Why? 


Echoing emergent calls to bring professionals into the study of IOs and 
global governance fields (Kauppi and Madsen 2013; Seabrooke and Hen- 
riksen 2017), we find surveys particularly useful in answering three sets of 
questions. Moreover, once built, survey-based datasets can then be dissemi- 
nated among researchers, either to make comparisons across populations or 
address new research puzzles. 


QI: What characteristics define 1O professionals as a population? 


Professionals, understood broadly as individuals who earn a living from their 
activities, are central protagonists in all forms of IO action, from diplomatic 
encounters to the design, implementation, and evaluation of international 
public policies. Though UN diplomatic and elite bureaucratic staff have 
received some attention in recent years,' IO professionals remain relatively 
understudied as a population. Capturing diversity and hierarchy within IO 
professional ranks, a survey can provide a macro-level description of an orga- 
nization’s workforce—the scope of which depends on sampling. Though 
not exhaustive, possible variables include: (1) personal information (i.e., 
nationality, age, civil status); (2) socioeconomic status (i.e., income, educa- 
tion, occupation); (3) professional experience (i.e., previous jobs, contrac- 
tual status, job satisfaction); and (4) ideational values and preferences (i.e., 
economic/political/religious/moral attitudes). Additionally, datasets can be 
compared and contrasted to assess the (non)specificity of IO profession- 
als in relation to other populations. For example, to understand the social 


1. Most recently, see for example Pouliot (2016) and Yi-Chong and Weller (2018). 
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construction of international peacebuilding, Goetze (2017) used web-based 
survey (WBS) methods to describe the experiences, professional trajectories, 
political values, and working environments of UN peacebuilders. 


Q2: How are socio-professional hierarchies within IOs structured? 


Building from descriptive data on IO professionals belonging to different 
hierarchical grades, surveys can be used to shed light on socio-professional 
hierarchies in IOs. By collecting data on professional mobility and status 
alongside socioeconomic indicators, we are able to map status and inequality 
in relation to social class, gender, nationality, profession, organizational loy- 
alty, and so forth. Survey data reveal what resources are necessary to succeed, 
professionally, in an IO setting. Hierarchical grades can potentially be ana- 
lyzed in relation to conflicts of interest, as in the case of Mele, Anderfuhren- 
Biget, and Varone’s (2015) study on what determines whether international 
civil servants defend the national interest of their country of origin or the 
international interests of the IOs they work for. 


Q3: How do IO professionals’ preferences and attitudes help explain 
international public policy dynamics? 


‘The intensified pace of globalization has led global governance to adapt, 
both in the norms and policies it produces and the way governance is prac- 
ticed, to meet challenges raised by the increased interconnectedness of 
economies and societies. Could some policy shifts and/or policy inertia be 
explained by individual or collectively shared preferences? In giving heed to 
the subjectivities of IO professionals, a survey can build a transversal con- 
nection between the individual, the organizational, and the systemic level. 
Analyzing survey data on employee job satisfaction from an organizational 
management perspective, Giauque, Anderfuhreng-Biget, and Varone (2016) 
have, for example, shown how individual perceptions of stress and turnover 
intentions can potentially impact IO performance. 


How? 


To give a brief overview of how to conduct survey-based research, we address 
five major considerations. 
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Selecting, Wording, and Ordering Survey Questions 


Overall, a good survey proposes a limited number of coherently formulated 
and structured questions. Three issues deserve particular attention: how to 
select, how to word, and how to order questions. 

Choosing which questions to include in a survey depends on the core 
research focus. Based on our research objectives, our questionnaire incor- 
porates two main types of questions: 1) questions specific to the humani- 
tarian field—i.e., What is your main motivation for working in the humani- 
tarian sector? How many years have you worked in the humanitarian sector?; 
and 2) general questions about participants’ socioeconomic status, beliefs, 
and values—i.e., How important is it for you to live in a country that is gov- 
erned democratically’. Regarding the latter, we have drawn questions from 
the World Values Survey (WVS) and the European Values Study (EVS), 
both recognized for their scientific rigor—a strategy also deployed by Goetze 
(2017). Within both surveys, we sampled questions with a demonstrated 
predictive and convergent validity over time, meaning they were replicated 
several times from 1981 to 2008 to control for survey-related biases such as 
social desirability’ or recall bias.’ 

In terms of wording, open-ended and closed-ended questions ideally 
need to be balanced, unless survey research is coupled with, for example, 
qualitative interviews, thereby reducing the need of open questions. Closed- 
ended questions are easier to analyze but may come off as irrelevant or over- 
simplified to a respondent. One way of overcoming this difficulty is to build 
on other surveys and organize a survey test run before officially administer- 
ing it. 

Finally, question order is of critical importance in designing a survey as 
preceding questions have been found to have a strong influence on answers 
given to subsequent ones. The first section is composed of icebreaking ques- 
tions that allow respondents to smoothly enter into the survey's topic. Start- 
ing with descriptive questions about the respondent’s current position or 
employment history, for instance, can be good icebreakers as they are fairly 
uncontroversial and easy to answer. The following sections transition to the 
core focus of the survey. Lastly, the ending section often offers respondents 
the possibility to comment on their perception and evaluation of the rel- 
evance of the survey’s questions. More difficult or intrusive questions regard- 
ing, for example, a respondent's political ideology or religious beliefs can be 


2. The tendency of the respondent to provide answers that appear as socially accepted even 
if they do not fit with their own opinions. 
3. The difficulty to recall past experiences. 
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put at the end, with the hope that they will not dissuade the respondent 
from finishing the survey. 


Survey Length and Completion Time 


Survey length and the articulation of questions require careful consideration 
and compromise. With many researchers falling into the trap of asking too 
many questions, a good strategy is to start with an exhaustive list of questions 
followed by a reassessment of priorities and compromise to then shorten the 
list. This strategy could however become more complicated as the number of 
active research partners (and competing interests) increases. 

Researchers also need to be attentive to the amount of time it takes to 
complete the survey, as time-consuming surveys run the risk of discour- 
aging participation. When respondents are not remunerated, we advise an 
approximately twenty-minute WBS. Keeping completion time under thirty 
minutes is particularly important for professionals working in high-pressure 
environments who are persistently pressed for time. Face-to-face surveys are 
also an option and can last longer as IO professionals are more accustomed 
to being interviewed for forty-five to sixty minutes. 


Sampling Techniques 


Often in IO research, scholars are dealing with spatially dispersed and partly 
undetermined populations, making sampling a challenge. In our case, our 
interest in assessing diversity drove us to prioritize a global sample of aid 
professionals working in diverse locations. But how exactly can we go about 
sampling? Multiple pathways exist, though we view the following two sam- 
pling methods as the most ideal: 


(1) Quota sampling. Use an available dataset cataloguing relevant 
organizations, such as the Yearbook of International Organi- 
zations dataset (UIA 2019), or a self-made dataset, such as the 
Humanitarian Organizations Dataset (Egger and Schopper 
2022) when a ready-made one is not available to then select a 
representative sample of organizations. Survey distribution will 
subsequently be negotiated with each chosen organization. 


(2) Convenience sampling. Use readily available mailing lists, such 
as the Professionals in Humanitarian Assistance and Protec- 
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tion (PHAP)* mailing list, or social media groups, such as the 
“International Humanitarian and Development Professionals” 
LinkedIn group. 


Distribution Strategies 


Depending on the selected sampling technique, several distribution tech- 
niques are possible, each with advantages and trade-offs (see table 4). 

At the end of the day, researchers should weigh the pros and cons of each 
distribution strategy to arrive at a decision that is most compatible with their 
interests and resources. 


Data Analysis 


‘There is no specific method to analyze survey data per se. Relevant qualita- 
tive and quantitative options need to be identified depending on the num- 


Table 4. Surveys and Distribution Techniques: Advantages and Tradeoffs 


Distribution Techniques 


Direct access through orga- 
nizational actors 


Indirect access through 
networked intermediaries 


Indirect access through 
social media platforms 


Advantages 


Robust, organization- 
wide sample; Nourish 
practitioner-academic 
exchange; Inexpensive; 
High response rate 


Less time consuming; Inex- 
pensive; Ability to reach 

an organizationally diverse 
population 


Less time consuming; 
Inexpensive; Easy to dis- 
seminate; Ability to reach 
an organizationally diverse 
population 


Tradeoffs 


Time consuming; Risk of 
access refusal; Potential 
intervention in research 
design; Biased responses due 
to feeling of organizational 
surveillance/control 


Biased sampling; No control 
over respondent selection; 
Low response rate 


Biased sampling (exclu- 
sion of individuals not on 
social media or in social 
media groups); Inability to 
isolate “nonhumanitarian” 
respondents 


4. PHAP is a member-based network of self-declared humanitarian professionals allegedly 
made up of 10 percent of the global population of aid workers, open to individuals with at 
least two years of “relevant experience” in the humanitarian sector. 
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ber of respondents, the sampling technique used, and the question design 
(i.e., open-ended or close-ended) and the overall objectives of the research 
(explore the attitudes of IOs staff, identify their determinants, or draw infer- 
ences). Based on these criteria, several strategies can be identified. As in 
qualitative content analysis, open-ended questions are best analyzed using 
deductive or inductive coding techniques. Descriptive statistics can be used 
to analyze answers to close-ended questions, focusing on a small sample of 
IO staff. Closed-ended questions collected on a large sample, however, are 
more amendable to methods used in statistical analysis, such as cross tabu- 
lation or regression analysis, to establish intervariable relationships or pin- 
point patterns and probabilities. The exact type of analysis to be performed 
depends on the response scale used (i.e., nominal, ordinal, interval, or ratio). 


What Challenges? 


Just as with every method, survey-based research necessarily involves a num- 
ber of challenges. 


Sample Representability and Survey Reliability 


No matter the sampling method, determining the representability of the 
sample population and the reliability of survey results is a major obstacle in 
survey research. Due to the absence of macro-data on humanitarian profes- 
sionals, a gap that our research aims to fill, and the fluid boundaries of the 
sector, it is impossible to get an a priori sense of sample representativity. 
Our inability to measure the representativeness of our sample population 
then makes it difficult to assess the reliability of our results. The best way of 
dealing with these challenges is to avoid overgeneralizing, while condoning 
an iterative approach whereby the accumulation of knowledge will serve to 
improve future survey studies. 


Access 


Gaining access to a global population of professionals, characterized by a 
strong degree of geographic and institutional fragmentation, is a compli- 
cated endeavor. The essence of this fundamental problem stems from the 
fact that we are unable to contact potential respondents directly, meaning 
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we must either seek formal authorization from an IO that will distribute 
our survey to their staff or operate through an intermediary. Inherent to the 
study of transnational professional populations, our only resolve is to accept 
these access constraints. 


Survey Language 


Working on a transnational population that is exceptionally diverse in terms 
of language and culture makes the survey’s language a matter of concern. In 
light of our respective linguistic capacities, we initially started with an English 
and French version of the questionnaire. However, our fear is that participants, 
especially entry-level national staff, might not feel comfortable enough with 
either. In order to ensure participation across hierarchical levels, one strategy 
is to translate the survey into the four other official working languages of the 
UN. If, at the analysis phase, a low response rate among low-level national 
staff is observed, a follow-up strategy might be to pick two or three country- 
specific cases, translate the survey into the dominant local language, and redis- 
tribute the survey. Although translation comes with its own costs and issues, it 
would help to offset nonparticipation due to language barriers. 


Question Inflexibility 


Surveys are, by nature, definitive. Unlike in semistructured or unstructured 
interviews, the researcher cannot experiment with the way they formulate 
their questions or adjust their interview guide in a piecemeal fashion. If you 
realize a question is incomprehensible or creates confusion once the survey 
has been diffused, your only option is to throw out the first set of responses 
and start from scratch. However, this might be costly and potentially com- 
promise your reputation and legitimacy. The best way of dealing with, or 
rather avoiding, this challenge is to spend sufficient time proofreading and 
testing survey questions. Using questions from high-quality surveys may 
also be a good strategy depending on the focus of the research. 


To Go Further 
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BOX G 


Languages and Interviews 


James Worrall 


Interviewing in your mother tongue presents enough challenges when 
engaging in research within international organizations—from register, 
framing, and specialized language (jargon), through to neologisms, contrac- 
tions (acronyms), and different techniques (Miles 1970). Interviewing in a 
foreign language is a challenge of a different order of magnitude. There is 
much to think about, from the philosophy of language itself to the practi- 
cal minutiae, in order to ensure that you are understood and that you truly 
understand the nuances within responses to your questions. 

International organizations are usually multilingual in orientation— 
even when one or two languages become dominant for day-to-day work. 
Some IOs such as the Arab League work only in Arabic, although many staff 
and diplomats also have proficiency in English or French. Despite officials 
often having command of multiple languages, it makes sense to interview 
someone in their own language if at all possible. 

There is a clear relationship between thought and language, so it is 
important to be aware of the nuances of how language is used, because as 
Slavoj Zizek suggests, “word is murder of a thing” (1991: 59). Many lan- 
guages have multiple graduated words that can have considerably different 
meanings. C’est compliqué in French appears to simply mean “it’s compli- 
cated.” Yet in some contexts its meaning stretches as far as saying it’s “over- 
sophisticated” or even “perverse.” Therefore the phrase pas si simple is used 
to get across the English meaning of “it’s complicated.” This example points 
to possible confusion exacerbated in IOs. As complex evolutionary cross- 
cultural spaces, they develop their own meanings for common words and 
ideas in the languages in which they work, such as the invention of the word 
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“comitology” for the work of committees within the EU, or the use of actual 
to mean current (Fredriksson et al. 2006). Researchers conducting interviews 
must acquire a sound awareness of how the specific language is used, as well 
as how it has evolved within an IO and be mindful of the ways multilingual 
staff even mix together different languages in the same sentences. This means 
that researchers should not be shy in seeking clarifications from interview- 
ees, even if interviewing in their mother tongue! 

‘These issues mean that the researcher should question their linguistic 
abilities—what might get lost in translation? Balancing the benefits of inter- 
viewing someone in their mother tongue with the level of fluency required 
to avoid loss of vital nuance might therefore necessitate working with an 
interpreter, which can cause its own potential issues. 
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BOX H 


Focus Groups 


Marie Saiget 


Focus groups can be defined as a collective interview of a small number 
of participants in a formal or informal setting. This method allows the 
researcher to understand perceptions, ideas, and feelings, to analyze narra- 
tives on/in world politics (Stanley 2016) and to investigate the representa- 
tions and strategies of the populations targeted by international organiza- 
tions (IOs) (Saiget 2015). However, it is not a standardized practice: choices 
vary when it comes to participant recruitment (people from the same inter- 
knowledge group or who do not know each other), interview facilitation 
(directive, flexible, role-playing games), and analysis (exhaustive analysis or 
focus on sequences; interpretative and qualitative, or quantitative analysis) 
(Garcia and Haegel 1996: 392). Neither does it have the same functions 
(basic or applied research uses; as a therapeutic or social intervention tech- 
nique) (Frey and Fontana 2019). Despite these differences, focus groups all 
value collectively constructed discourses: interaction among participants is 
seen as a productive setting to study shared views as much as discrepancies 
in a defined group. 

While focus groups are used by researchers, they are also by IOs and 
nongovernmental organizations (NGOs) to collect data on the perceptions 
of the “beneficiary” populations to evaluate the impact of their programs or 
give them insight into the social realities on the ground. If researchers can 
benefit from IO focus group practices to gain greater access to the groups 
targeted by IO programs and their perceptions, the quality of the data one 
collects may for several reasons not always be satisfying. In my research expe- 
riences with the United Nations Refugee Agency (UNHCR) in Cameroon 
(2010), on the one hand, and CARE International in Burundi (2013), on 
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the other, the process constrained my choices both in recruiting participants 
and in developing methods of data collection. Furthermore, the conditions 
of the focus groups were strongly affected by the translators’ skills (the field 
coordinators whose competence were limited) (see box g—Languages and 
Interviews), so much so that participants repeated the same story time and 
again. The story resembled the “success story’—a narrative put forward by 
the NGO (the “Set Speech” obstacle; see box i—J/nterviewing “Beneficiaries’). 

Yet in Burundi, this hurdle became an opportunity that proved to be 
insightful on the methodologies employed by CARE International and the 
context of interaction between aid workers and the populations targeted by 
the NGO’s interventions. Analyzed reflexively, these focus groups provided 
less accurate knowledge of the realities of the persons I met than informa- 
tion about how these audiences conformed to the NGO’s expectations. They 
revealed the influence of this technique on the way individuals performed 
the role of “beneficiaries.” In addition, participants in the focus groups were 
able to express requests of various kinds that were not necessarily related 
to the project, thus instrumentalizing focus groups to advance their own 
agendas. 

To conclude, using focus groups to understand IO practices has advan- 
tages for several reasons. By using their own practice in a mirror effect, it 
highlights the complex social interactions between IOs and their “benefi- 
ciaries.” It also shows the power and limitations of this technique as a mode 
of intervention and knowledge production on the social realities IOs intend 
to change. 
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CHAPTER 5 


Semistructured Interviews 


Mélanie Albaret and Joan Deas 


Carefully prepared and used, semistructured interviews are a valuable 
method of data collection and data triangulation. When researching inter- 
national organizations (IOs) in which informal and closed processes are fre- 
quent, they are sometimes the only way to obtain information. 


What? 


The semistructured interview is a qualitative research method that con- 
sists of collecting data on facts and representations discovered during oral 
exchanges. It is meant to seek out the world views of respondents in a flex- 
ible manner, in relation to a fixed research objective (Bryman 2012). Semi- 
structured interviews establish a social relationship in order to induce direct 
interaction with another individual, who is simultaneously the researcher's 
interlocutor and subject of study (Allès et al. 2016: 159). 

Scholars traditionally distinguish them from close-ended/structured 
interviews (the same questions are asked in the same order and the inter- 
viewees must choose their answer from a limited number of responses), and 
open-ended/unstructured interviews, which proceed freely from an initial 
question. Semistructured interviews also encompass a variety of practices, 
from ethnographic interviews (see chapter 3—Ethnographic Interviews; 
Spradley 1979; Beaud and Weber 1997) to focused interviews (Merton et 
al. 1956) or in-depth interviews (Bryman 2012). 

With the opening of international relation studies to other social 
sciences—including qualitative sociological studies—in international rela- 
tion studies in the 1980s, semistructured interviews have become a com- 
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mon qualitative method to analyze international organizations. It enables 
the researcher to enjoy the benefits of a direct, in-person exchange with 
those highly mobile and hard-to-reach populations (IO staff, national dip- 
lomats, experts, etc.), who would otherwise be cross-nationally dispersed 
(Efrat 2015). Scholars who have studied IOs from various perspectives, such 
as from that of the European defense policy (Mérand 2008), IOs as interna- 
tional bureaucracies (Bierman and Siedenhiiner 2009), the role of states in 
a specific institution (Freedman and Lemay-Hébert 2019), and the role of 
experts in IOs (Littoz-Monnet 2017), all chose this method. 

In the literature (e.g., Rubin and Rubin 2012; Kvale 2012; Gubrium 
and Holstein 2002), a set of good practices seems to prevail about four par- 
ticular potentially challenging issues related to semistructured interviews: 
access and logistics issues (entering the IO “sanctum,” bridging the insider- 
outsider gap, sampling frame, networking, and time and cost issues); ethics 
and confidentiality (Informed Consent Forms [ICF], recording, anonymity, 
culture of secrecy, trust and intimacy issues); power dynamics (researching 
the powerful, building and sticking to interview guides, positionality, social 
status and gender issues, control and manipulation); data analysis and use 
in the final work (quality/reliability/validity, transcribing, triangulating and 
verifying, coding/CAQDAS/quoting). 

While the exercise appears specific and delineated, it is not always easy 
for students and scholars to find their way as the practices, uses, and presen- 
tations of interviews fluctuate, partly due to the uncertainties the exercise 
generates (Kvale 2007: 8; Platt in Gubrium and Holstein 2001: 51) and to 
the debates interviews have triggered in social sciences (e.g., Bourdieu 1993; 
Mayer 1995). This chapter provides insight on frequently asked questions 
and steps taken in the semidirective, or semistructured, interview process 
while studying IOs, and not meant as a widely applicable formula. 


Why? 


Semistructured interviews can be used in a variety of ways depending on 
cases and research temporality. 

First, exploratory semistructured interviews aim at constructing, speci- 
fying, and refining the research object and opening up the field of inves- 
tigation, which is often difficult when it comes to IOs (Abélés 2011: 22). 
Exploratory interviews enable researchers to question the feasibility of the 
research and assess the physical access to the IO, which helps design an often 
costly and time-consuming “entry strategy.” 
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Second, semistructured interviews are also of an informative nature. 
Because a large part of IOs’ work is based on informal discussions, oral 
exchanges, and behind-the-scene operations, semistructured interviews are 
often the only viable conduit to discovering factual data, accessing certain 
types of information, and reconstructing a course of events or decision- 
making processes. They proved to be relevant for the study of hard-to-reach 
fields, such as the humanitarian space in Somalia (Egger 2016) or the UN 
Security Council (Adler-Nissen and Pouliot 2014). 

Third, semistructured interviews can have a more comprehensive dimen- 
sion because they allow researchers to get a good grasp on practices and 
representations, and to understand networks of alliances and interpersonal 
enmities and affinities between and within the IOs’ actors. Through inter- 
views, Vincent Pouliot was able to make sense of the codes and implicit pro- 
cesses of the multilateral diplomatic cliques at the UN and NATO (2016). 

Last but not least, semistructured interviews are often combined with 
other data collection methods (such as archives, grey literature, or partici- 
pant observation). Interviews are of a complementary nature and help trian- 


gulate other sources or interviews and contextualize facts, speeches, or obser- 
vations made by the researcher (e.g., Guilbaud 2015; Adler-Niessen 2013). 


How? 


Once preliminary work has been completed (research question and objec- 
tive, readings), the interview subjects are to be identified. While statistical 
or probability sampling best serves research topics using quantitative meth- 
odology, purposive sampling is the most common method used in qualita- 
tive studies (Bryman 2012). It includes general, snowball, and theoretical 
samplings. IO actors are not homogeneous in their behaviors toward the 
researcher, and the response rate can fluctuate immensely. Thus the choice 
of sampling method often needs to adapt to empirical constraints: access to 
and familiarity with the field, respondents’ status, culture of secrecy, research 
topic’s sensitivity, and so forth. 

Although a lot of contact information can be accessed through various 
open-source mediums such as the UN Blue Book, permanent missions 
(PMs), embassies, consulates, and UN websites (Albaret and Placidi-Frot 
2018), snowball sampling (sometimes complemented by contact with a 
key informant) often appears as the best-suited method to access “hard-to- 
reach” respondents in “closed,” secretive, and/or sensitive fields such as IOs 
or PMs. Relying on respondents’ own personal and professional networks 
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allows the researcher to capitalize on trust and better bridge the insider/ 
outsider gap. 

Scholars can use different methods to contact interviewees depending 
on their identity (nationality, seniority, rank) and experience. Email, phone 
calls, WhatsApp, headed letters, and physical requests on site can be used 
simultaneously or alternatively. Researchers should be “politely persistent” 
(Aberbach and Rockman 2002: 673), have a short written and oral presenta- 
tion of the research ready for use, and be prepared to repeat it on many occa- 
sions to various intermediaries. Contacting the respondents in their mother 
tongue when mastered, “getting a badge” to access the facilities in order to 
“know the house” and be seen around, and/or relying on institutional cred- 
ibility and prestige usually facilitate access. 

A flexible semistructured interview guide identifying a limited number 
of “essential” questions should be tailored to the specificities of the IO field 
and the targeted people (staff categories, diplomatic rank). Open-ended 
questions and asking “how” and not “why” (Becker 1998: 58) usually gener- 
ate better receptivity from respondents. Icebreaker questions help establish 
a trusting environment and convince the participant to answer more sensi- 
tive questions later on. Since some IO staff and diplomats are professional 
communicators, it may prove challenging to frame the discussion, refrain 
from being carried away from the main topic, or interrupt the respondent. 
Hence the interview guide may contain “structuring” questions to rekindle 
an interview, along with “follow-up” and “specifying” ones to get the respon- 
dent to elaborate his or her answer (Bryman 2012: 476). 

The relationship that is established during an interview is seldom char- 
acterized by full confidence and symmetry. ICF (which has different status 
depending on the country and the university—ranging from compulsory 
to not required) is supposed to reassure the interviewee by providing him/ 
her with a guarantee of legal protection in case of breach of confidentiality. 
Specifying the rules of anonymity and agreeing on either the possibility or 
the interdiction to record the interview also help build confidence. 

Recording “facilitates the use of a conversational style and minimizes infor- 
mation loss” (Aberbach and Rockman 2002: 675). Not recording, however, 
can sometimes have a positive impact as it provides more freedom to respon- 
dents. Whether or not the interview is recorded, the researchers must take 
handwritten notes. A field journal should be established to recall the con- 
text and details of each interview (Beaud and Weber 1997). As the exercise 
is intense, limiting the number of interviews per day and freeing up time for 
transcription the same day to avoid loss of information are essential practices. 

‘Transcription is a retrospective translation of the spoken into the writ- 
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ten word. It is mandatory to enable a systematic analysis and scientific use 
of the data collected. It allows more thorough and repeated examination 
of the respondents’ answers, enables coding and quoting, helps correct 
the researcher’s biases, and opens up the data to public scrutiny by other 
researchers. Depending on the nature of the study (ie., discourse analy- 
sis versus content analysis), the transcription concerns the whole interview 
(taking into account silences, hesitations and verbal tics), or sections when 
large portions are not useful to the research. (Bryman 2012). The data col- 
lection is then transformed in an academic analysis through data handling 
and interpretation. By coding the interview transcriptions, scholars organize 
data, compare them and go beyond description by integrating them into a 
conceptual and/or theoretical analysis (Gibbs 2018). 


What Challenges? 


Although we have already considered several potential difficulties, two issues 
appear to be particularly challenging. 

Power imbalance, also referred to as “double trouble” (Welch et al. 1999: 
3; see box j—Asymmetrical Interviews) can be further exacerbated by the 
researcher's social status (gender, age, citizenship, institutional affiliation, 
etc.). The asymmetry between the interviewer and the respondent can also 
be reversed, when respondents are for instance young, on precarious con- 
tracts, are locals or interns . . . (see box i—Interviewing “Beneficiaries”). In 
order to limit the impact on the data collection process and avoid potentially 
harmful and ethically questionable experiences, scholars must reflect on the 
“interviewer effect” (Hyman 1954) and pay attention to the presentation 
of oneself and one’s research project. Having prior knowledge of protocol 
and getting an idea of common social practices help define the positions 
one can (or cannot) take as an interviewer. For instance, the definition of 
the adequate place for an interview differs according to the respondent's 
identity. Interviews with precarious or vulnerable IO staff would take place 
under better conditions out of the office. Interviews in the field respond to 
still other logics (Dauvin and Siméant 2001). 

Another objection to semistructured interviews is that the collected data 
would be “subjective and imprecise” (Rathbun 2008: 685). The method 
would lack rigor. Using semistructured interviews is also costly and time 
consuming. The data obtained must be processed with care. These chal- 
lenges present sharply in IO studies due to the contrast between the high 
staff turnover and the important circulation of diplomats and state repre- 
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sentatives (which aggravates issues of cost and time) and the small circle of 
multilateral activity (that strengthens the secret imperative for most of its 
actors). Very restrictive anonymity requests can complicate the use of the 
interview content, raising the question of the relevance and even possibility 
of the data use. 

To overcome these challenges, the researcher could opt for a modest 
approach by considering semidirective interviews as complementary to other 
data collection sources, and in order to triangulate data. Holding multiple 
meetings and interviews with the same respondent optimizes data quality 
and builds trust. Another way to overcome the challenge of data reliability 
would be to study the same IO for a long time. It would then build memory 
that may be lacking among the authors’ interlocutors, providing researchers 
with more perspective on the data collected in semidirective interviews. In 
any case, a clear explanation and contextualization of the researcher's meth- 
odological choices and constraints needs to appear in the final research. 


To Go Further 
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BOX I 


Interviewing “Beneficiaries” 


Lucie Laplace 


‘This box focuses on interviewing IO “target population”: individuals who 
receive assistance in various forms. IOs select “beneficiaries” according to 
their perceived level of vulnerability and the characteristics of the local 
context (Corbet in Ribémont 2016). Scholars must reflect on the so-called 
beneficiaries’ identity construction, the challenges of such categories, the 
resources they provide and the mobilizations they produce. Interviewing 
these individuals requires a reflexive posture to consider and mitigate the 
unbalanced relation—to the detriment of the informant—which has con- 
sequences on the data interview. Depending on the research question, the 
researcher must choose the most suitable methods of investigation (survey 
interview, semistructured interview, life trajectory, repeated interviews, 
collective interviews, or focus groups), while identifying a relevant place 
(answers may differ if the interview is conducted in the IO building, a public 
space, or another place chosen by your respondent). At the beginning of the 
interview, you may explicit your link to the IO. After agreeing on the anony- 
mization process, you may ask them to choose another name. There are two 
main challenges in interviewing “beneficiaries”: (i) getting access to them 
depending on institutional constraints and (ii) the “Set Speech” obstacle. 

IOs often have a restrictive communication policy (such as the signature 
of a nondisclosure contract), both with the researchers and their “beneficia- 
ries.” There are two coping investigation strategies: (i) meeting IO require- 
ments and (ii) reaching directly the beneficiaries in intermediate safe spaces 
(through partner organizations or in the public space). For example, since 
the UNHCR did not respond to my requests to connect with the beneficia- 
ries, I implemented an alternative strategy, namely attending NGO work- 
shops and public events for refugees. 


90 


Interviewing “Beneficiaries” 91 


Some respondents are used to adapt their speech within the institutional 
framework creating a form of “Set Speech” obstacle. Their testimonies are 
therefore close to those given to the organization, which then provides infor- 
mation about the IO functioning. If the field is “saturated,” respondents 
may also show fatigue and exasperation signs (Clark 2008). After several 
meetings, you may encourage respondents to show less reticence (with- 
out making them lose face as beneficiaries and consequently threaten their 
resources). To do so, you may display skepticism in order to encourage them 
to raise the limits of the organization and express their specific knowledge 
about the IO. Like in other interview contexts, make sure that the person 
feels comfortable: show empathy, flexibility, and proximity through cultural 
and/or linguistic knowledge while considering their daily life experiences 
and practices though “bottom-up” and “top-down” approaches to avoid 
populist bias (Olivier De Sardan 2005) and to identify the implications of 
the IO policies. 

Finally, you may also anticipate as much as you can about the possible 
spread of your field materials and of your analysis, at least by applying the 
“Do No Harm” principle (Hugman and al. 2011). 
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BOX J 


Asymmetrical Interviews 


Emilie Dairon 


As much as the researchers are prepared to undertake an interview with 
informants on international organizations (IOs), it may fall off their hands 
and take a totally unexpected turn. This can happen especially with “domi- 
nants,” i.e., high-level officials such as directors-general, ambassadors, and 
so on—to paraphrase Conti and O’Neil (2007: 63)—people who inhabit 
the highest strata of the IO world. What has been studied in elite research 
holds true in interviews with IO leaders: since interviews are social interac- 
tions, the power asymmetry between researcher and interviewee can lead to 
an unbalanced relationship. The sense of “losing control” in elite interview 
is well documented (Conti and O’Neil 2007). Its manifestations include the 
interviewee wanting to control the frames of interaction (location, condi- 
tions; Laurens 2007), wanting to control the frames of experience (enhanc- 
ing seniority and position of power; Laurens 2007), or just not showing 
commitment (Kezar 2003). 

It happened to me once with an IO leader. It was unexpected: I already 
interviewed this person a year before and this encounter was cordial and 
informative. The second interview was undertaken jointly with a coauthor 
and we immediately felt that it was not going as we expected. The inter- 
viewee seemed annoyed to have to spend time with “students” (as his sec- 
retary announced us) between two meetings with ambassadors. During the 
discussion he was deciding everything: the language, the topics, and their 
order. We left the room after thirty minutes of discussion, with the feeling 
of a failed encounter. 

‘This experience led me to think of adaptation strategies, and I see three 
moments when mitigation plans can be implemented to avoid this experi- 


92 


Asymmetrical Interviews 93 


ence. The first one would be a preventive strategy: before the interview, make 
sure that the conditions are gathered for a fruitful interaction. Write your 
introduction (e.g., email or letter with official logo of your institution) care- 
fully, paying attention to your title (prefer “researcher” to “applicant”). Like 
other interviews but with an extra care, prepare your questions in advance; 
if possible, go and check the location in order not to be late or impressed by 
the building. A second mitigation strategy is to try to retake the lead dur- 
ing the interview: either turn the “pillars of intimidation” into assets (e.g., 
talking about the interviewee’s impressive results), or place yourself into the 
expert position, in order for the informant to see you as a peer (Kezar 2003) 
or to consider the interview as gratifying (Laurens 2007). Finally, if despite 
these strategies the interview still leaves you with a bitter taste, do not reject 
its outcomes because of a feeling of despondency (Conti and O’Neil 2007). 
From an epistemological point of view, it is very informative to analyze why 
this specific interaction was “odd.” A final advice would be not to be afraid 
of interviews with IO elites: most of the discussions with high-ranking indi- 
viduals are fruitful; and in any case, combining interviews from several levels 
of hierarchy is a good strategy to approach your research object. 
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CHAPTER 6 


Biographic Interviews 


Monique J. Beerli 


Every day, thousands of people across the globe get up, get dressed, and go 
to work . . . for an IO. In fact, IOs would not have much agency beyond 
the parchments that establish their rational-legal authority if it were not for 
privileged and precarious individuals acting in their name. As Yi-Chong and 
Weller poignantly remind us, “IOs are not only the world of powerful states 
but also a world of many ‘puzzled’ people whose daily life is to assess options 
and make choices” (2018: 7). To enter into the situated, lived, and imagined 
worlds of individuals that “make up” IOs, this chapter offers up biographic 
interviews, or life stories, as a methodological lever. 


What? 


Grouping together a wide range of approaches from autobiography and 
autoethnography to oral history and life history, biographic research is char- 
acterized by its unrivaled concern with “researching lives and the stories 
people tell about them” (Merrill and West 2009: 2). Through direct and 
indirect observation, biographical research reconstructs an individual's life, 
past, present, and foreseen future, recognizing the agency of individuals in 
creating meaning while capturing their dynamic interplay with sociohistori- 
cal forces. Combining the objective and subjective, in-depth interviews rep- 
resent just one of several tools amenable to biographic research. 

By default firsthand, retrospective narratives lured out of subjects 
and coproduced with an inquirer, biographic interviews—or life story 
interviews—have a scattered disciplinary past. Now prominent in a wide 
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range of fields, biographic interviewing is most deeply rooted in the meth- 
odological worlds of history and sociology. Slowly since the mid-twentieth 
century, the conjuncture of multiple intellectual moves—from the advent of 
social constructivist theories to the rise of postcolonial and feminist thought 
to a revived historical interest in the everyday—has led to a “biographical 
turn” in the social sciences. Creating a space for the historically undocu- 
mented and the socially marginalized to be weaved into or against grand 
narratives, this biographical resurrection has enabled a more pluralized, mul- 
tilevel view of politics, economy, and society. 

As far as IOs go, biographical approaches are on the rise, with research- 
ers turning to interviews to reconstruct the career trajectories, untangle the 
complex identities, and trace the practices of the individuals embodying 
IOs (Emmerij et al. 2005; Pouliot 2016). Reflections on the epistemology, 
practice, and ethics of biographic interviewing in IOs and global governance 
research, however, remain rather scant (Roth 2015: 9; Dezalay and Garth 
2002). 


Why? 


While the expansion of IO studies as a subfield has been coupled with an 
undeniable pluralization of methodological research strategies, as this hand- 
book clearly corroborates, biographical interviewing continues to occupy 
a marginal position. Yet if we take seriously the claim that the lived expe- 
riences, trajectories, and worldviews of ordinary individuals constitute the 
sociological flesh and bones of IOs, life stories act as a powerful instrument 
for accessing micro-realities that ultimately shape global governance. 

Though biographic interviewing need not be exclusive, leaving the possi- 
bility of incorporating biographic questions into a more traditional qualita- 
tive interview, it is particularly adept at bringing the agency and subjectivity 
of individuals into the analytical foreground. Sensitive to the socializations, 
education, beliefs, career aspirations, experiences, perceptions, collegial rela- 
tions, and personal relationships of IO professionals, biographic interviews 
offer insights into micro-level dynamics that condition meso- and macro- 
level phenomenon, such as change and stability in IOs or transformations in 
international order. Endowed with the potential to disrupt dominant onto- 
logical framings of IOs and pluralize global governance lifeworlds (Roth 
2015), biographic interviews perform the dual task of disruption/recon- 
struction through four types of enactments: personifying, informalizing, 
historicizing, and unclosing. 
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Personifying IOs. Close your eyes and imagine the United Nations. What 
comes to mind? Perhaps an azimuthal projection of the globe cradled in olive 
tree branches, long rows of state flags, or the newly renovated Security Council 
chamber for the UN connoisseur. But what about the UN workforce? Like the 
unidentifiable civil servant holding a briefcase portrayed in Magnus Témas- 
son’s sculpture Unknown Bureaucrat,' the IO workforce is, for the most part, 
faceless (Reinalda, Kille, and Eisenberg 2018). By asking interviewees about 
personal matters, such as their youth or what motivated them to pursue an 
international career, biographic interviews give the means to personify and 
deimpersonalize IOs (Emmerij et al. 2005; Yi-Chong and Weller 2018). 

Informalizing IOs. On paper, IOs appear as disembodied, impersonal- 
ized rational politico-bureaucratic machines governed by formal rules, pro- 
cedures, and processes. In principle, subordinates report to their hierarchical 
superiors, organizational policies are respected, and professional miscon- 
duct is reprimanded. However, the reality of how IOs function is far more 
complicated, often left out of official records that build an appearance of 
coherence and order. Supporting a nascent impulse to study IOs not as they 
should function but as they do function (Littoz-Monnet 2020), biographic 
interviews open up insights into informal practices and relations that struc- 
ture the everyday life of IOs (Roth 2015). 

Historicizing IOs. Many individuals pass only fleetingly, for the dura- 
tion of a summer internship or a short-term consultancy, through the doors 
of an IO office. Others, however, spend years if not their whole careers 
there, like a fly stuck to the wall. Albeit to varying degrees depending on 
career trajectories, biographic interviews targeting the past work experi- 
ences of interviewees help fill in the historical gaps that official documents 
fail to convey or conveniently omit. Life stories force a dialectical rereading 
of formal documents as primary registers of IO activities, as they nuance 
and complicate grand historical narratives about IOs and global governance 
(Reinalda 2021). 

Unclosing IOs. Given their focus on individuals or groups as primary 
units of analysis, biographic interviews provide the heuristic means to 
traverse or dismantle the organizational boundaries of IOs. If participant 
observation and interviewing have drilled holes into the edifices of IOs, bio- 
graphic interviews engage in acts of perforation but have the added feature 
of drawing transversal lines across institutional spaces to better capture the 
circulation of people, ideas, knowledge, and policies (see box r—Studying 
Ideas). Destabilizing the closed container image of IOs, life stories reveal 
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porosity by making visible “revolving doors” (Seabrooke and Tsingou 2020) 
between sectors, institutions, or scales classically conceived of as separate 
from one another. 


How? 


Lending themselves to multiple uses, biographic interviews can be con- 
ducted in a more or less structured manner, in formal or informal settings, 
multiple times over the course of an individual’s life or in a single instance, 
with or without a recording device. In comparison to more generic qualita- 
tive interviews, biographic interviews are particularly intimate, interaction- 
ist, and intrusive (Demaziére 2008: 19; Merrill and West 2009: 18). Focused 
on individuals and their lifeworlds, they create scenarios that will necessarily 
challenge one’s neutrality and are marked by asymmetric power relations 
(Demaziére 2008: 17-20). Given these specificities and lessons learned in 
my own research experience (Beerli 2018), there are at least three aspects 
of biographic interviewing that demand special attention: self-presentation, 
interview settings, and question framing. 

Presenting the self. Negotiating and maintaining entry into the lives 
of others passes through many unspoken forms of communication. As the 
impression you give will mold the type of relation you are able to establish 
(Demaziére 2008), be attentive to how to carry yourself. For starters, adapt 
your attire, dressing more or less formally based on your knowledge of the 
field and the nature of the meeting place. Charged thoughts may rever- 
berate in facial expressions and body gestures so try to suspend judgement 
when receiving narratives, practicing a strategy of empathetic listening. Sit 
in a relaxed fashion, maintaining direct and engaged eye contact, but with 
a softened, relaxed gaze. When possible, create parallels between yourself 
and your counterpart, but refrain from talking too much. Also, don’t be 
too discouraged when interviewees remain reserved or respond unkindly to 
questions, as IO professionals are not accustomed to speaking about work 
and their private lives with strangers. 

Setting the stage. Settings can have interactional effects so give some 
thought to the location of the interview, privileging sites that offer privacy 
and create an environment of ease, intimacy, trust, and release. In a first 
instance, let the interviewee propose a meeting spot, as they are likely to sug- 
gest a familiar place where they will feel comfortable. More often than not, 
you will be asked to meet at their workplace. Avoid meeting rooms or public 
office spaces such as a cafeteria or coffee corner, though, where exposure, 
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interruptions, and competition for shared spaces will create time constraints 
and favor self-censorship. Guaranteeing privacy while giving you a glimpse 
into everyday work life, well-concealed individual offices are more ideal set- 
tings. Outside locations, such as coffee shops or restaurants, are another 
possibility. Try to avoid meal-time meetings, especially for a first interview, 
though, as dining can be distracting. The most intimate of conceivable loca- 
tions, private homes will occasionally be opened to you, notably when dia- 
loguing with retirees or people with whom you share personal ties. When a 
face-to-face interview is not possible, opt for a video call over a voice-only 
call. If ever the setting is off-putting, no matter the location, just end the 
interview early and reschedule for another time, in another place. 

Cultivating moderate egocentricity. Invariably anthropocentric, bio- 
graphic interviewing encourages respondents to narrate events through their 
experiences (Merrill and West 2009). To cultivate moderate egocentricity, 
without awakening the narcissist or paralyzing the timid, opt for question 
formulations that emphasize human agency, even when talking about global 
politics and policy processes. If, for example, researching agenda-setting pro- 
cesses, ask your interviewee “what role did you and your team play in fram- 
ing women’s reproductive rights as a global issue?,” not “what role did UN 
Women play?” Having already forewarned your interviewee about your use 
of a biographic method, begin with questions such as “Can you tell me about 
your current role/position?,” progressively working your way toward more 
intimate discussions about life choices, events, and relationships. Without 
seeming insincere, end an interview by valorizing your interviewee’s story 
and thanking them for opening up. 


What Challenges? 


Incommensurate with other types of interviews, biographic interviews delve 
deep into the realm of the personal, the intimate, and the secretive. As they 
navigate through the dilemmas of infiltrating and writing on private spaces 
(Arthur and Kurvet-Kaosaar 2015), biographic interviewers will undoubt- 
edly encounter at least three challenges, whose consequential effects can be, 
at the minimum, partially mitigated. 

The missing, the fuzzy, and the false. Life stories are prone to be incom- 
plete and even slightly embellished (Bertaux and Kohli 1984). This makes it 
all the more important to gently steer interviewees to speak to life events that 
are of greatest interest to the researcher. Yet if there is a considerable tempo- 
ral distance between past experiences and the present or if the interviewee’s 
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memory is incomplete, important details may be forgotten and fuzzy. When 
confronted with gaps, try asking your interviewee the same question at a 
later time by email or over the phone; when possible, triangulate interviews 
with other sources (i.e., other interviews, organizational reports, grey litera- 
ture, etc.). But don't kill yourself over getting the “full story” as the devil isn’t 
always in the detail. Always slightly orchestrated, narratives might contain 
exaggerations, inaccurate accusations, intentional silences, and distortions. 
As opposed to confronting an individual about misrepresentations, consider 
such tales in relation to sensemaking processes and the relational positional- 
ity of the storyteller in a particular social space. 

Emotional debris, self-awareness, and oversharing. As interviewees dig 
into the personal and the intimate, making connections between their inner 
and outer worlds, they may be brought to confront emotionally charged 
situations from the past or dive into a state of self-awareness. For individuals 
in high-stress, demanding working environments, biographic interviews can 
offer individuals an opportunity to self-reflect on the paths they've followed, 
sometimes resulting in personal breakthroughs and expressions of gratitude. 
Playing an unlicensed therapist can be quite time-consuming and delicate, 
though, as interviewees are more likely to tangent and maybe overshare 
about psychological issues, relationships, and the like. When interviewing 
IO professionals such as humanitarians or peacekeepers who are hyper- 
mobile and/or traumatized by close encounters with death, destruction, and 
suffering (Roth 2015), the researcher may be confronted by heavy emotional 
debris. In such instances, move away from the position of an opportunis- 
tic, time-pressed data-hunter, recasting yourself as an empathetic listener. 
In moments of emotional intensity, let the conversation take its course. No 
matter how irrelevant things may seem, avoid rushing to your next question, 
which might leave your counterpart feeling vulnerable and used. Unsure of 
where a conversation might lead, it’s best to allow yourself ample time, say 
two or three hours, for each interview. 

Ethics. Approaching personal lives as gateways into IO and global gov- 
ernance lifeworlds comes with great ethical considerations and responsibili- 
ties. Whether or not imposed by a research ethics committee, ethical aware- 
ness should guide your practices throughout each stage of research—not 
just at the beginning or the end. Before beginning an interview, work out 
how the material can be used, navigating between your intentions and the 
interviewee’s concerns. When writing and preparing to publish, you will 
likely need to fully anonymize your interviewee’s identity to protect their 
privacy and avoid causing them or people they mention reputational and 
professional harm. This is particularly necessary when the “dark side” of IO 
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life had been exposed. Just changing or omitting an interviewee’s name wont 
always be sufficient for guaranteeing external as well as internal anonym- 
ity (Wiles et al. 2008). Depending on the scenario, you may have to avoid 
specifying for what organization, when, and where your interviewee worked. 
When deciding how to handle intimate details laid bare in an interview, be 
utilitarian, only divulging sensitive information when it is truly essential for 
your argument. 

Though not for every researcher, biographic interviewing—when done 
with sensitivity, respect, patience, and social awareness—offers a unique 
entry into the multiple realities of IOs through the perspective of the indi- 
viduals that incarnate them. 
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BOX K 


Online Interviews 


Leah R. Kimber 


Semistructured interviews hold no secret for social scientists in qualitative 
research: they capture participants’ experiences in their own words and reveal 
the context and meaning of their actions (Seitz 2016). If well-established, 
research partnership (researcher and participant) allows for a successful and 
detailed qualitative interview. Participants may share feelings, thoughts, 
experiences, and anecdotes while relying on trust (Seitz 2016). Because 
the interaction also relies on nonverbal communication, the literature— 
see Goffman and Garfinkel to mention the precursors—recommends 
face-to-face encounters. However, in some cases, “traditional” face-to-face 
interviews—the “gold standard” (Deakin and Wakefield 2013)—cannot be 
conducted. Dating pre-COVID-19 already, researchers have had to come up 
with viable alternatives to minimize the impact of time, space, and access 
(Janghorban et al. 2014; Deakin and Wakefield 2013). Online interfaces 
that provide simultaneous video calls appeared as the technological solu- 
tion to overcome travel costs and busy time schedules. Online interviews 
also allow research to become more inclusive by integrating participants in 
various locations, with various schedules. Recent publications (see Bourrier 
and Kimber 2022 for extensive bibliography) have addressed these meth- 
odological benefits, but also underlined their disadvantages, namely call 
disruptions, pauses, inability to read body language, and loss of intimacy, 
image blurriness, communication delay (Seitz 2016), or even the possibil- 
ity to withdraw from the conversation by clicking a button (Janghorban 
et al. 2014) or faking a disruption. For international organization research, 
working on the United Nations may imply giving up that “gold standard” 
and making online communication the sound alternative, in particular for 
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interviews with civil society members around the world. Though initially 
reluctant to using online platforms, I emailed my informants suggesting vir- 
tual interviews to avoid time and financial constraints—in hopes to increase 
participation thanks to greater flexibility (Janghorban et al. 2014). Surpris- 
ingly, my online interviews were more fruitful and detailed than face-to-face 
ones. First, in line with what I had observed at the UN, online communi- 
cation is a common tool in the organization’s daily practices, as it is free 
and easily accessible with Internet connection. Video calls thus reproduce 
face-to-face meetings where holding a phone is replaced with holding a tea 
mug. Second, virtual meetings were mostly scheduled on personal free time 
and carried out from home in the participants’ comfort space (Seitz 2016), 
making for a more intimate exchange. People were “hosting me” in their 
living room or office. Third, even if the interviewees might have known and 
trusted me, a sense of anonymity settled due to the screen, making it easier to 
express themselves in an uninhibited way (Janghorban et al. 2014). Fourth, 
online interfaces are not immune to communication disruptions or delays, 
the interviewee’s sense of urgency and the researcher’s discomfort to inter- 
rupt the participant’s thought process makes interviewees talk continuously 
making my role as interviewer almost secondary. I felt less compelled to 
follow-up on answers due to rare pauses. In sum, virtual meeting platforms 
for simultaneous interviewing processes should not only be considered as 
negative alternatives but rather as productive means to generate robust data. 
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INTERLUDE II 


Controversies in Interview Research 


Annabelle Littoz-Monnet 


Interviews are a “method” of data collection or generation that is today 
widely used in research on international organizations (IOs). What inter- 
views are; the way to handle the interviewing process; how best to analyze, 
evaluate, or interpret the interview “text”; and how to apprehend power 
relations during the interviewing process have been widely debated among 
scholars using interviews. Clearly, doing “interviews” in itself does not say 
much concerning a project’s research design, unless the questions above are 
also addressed. Yet existing insights in the literature too often solely con- 
centrate on the practicalities of research drawing on interviews, focusing on 
how to choose whom to interview, how to gain access, and how to estab- 
lish a good “rapport” with the interviewee. While relevant, such questions 
are often addressed without prior consideration of the epistemological status 
of the knowledge created through interviewing techniques, thus creating some 
confusion in methodological discussions. Diverging conceptions on the 
relationship between the interviewee and the interviewer have fed a number 
of controversies concerning interviews and their interpretation. First, such 
conceptions have informed competing conceptions of “validity” in relation 
to interview knowledge. Second, they have also resulted in debates over the 
kind of power dynamics that might be at play during interviews, and in the 
process of interpreting them. 


Debates on Validity 


Until the 1980s, most of the literature on interviews engaged with “how” 
to ask interview questions, how to avoid bias both during the interview as 
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well as during the process of analysis of the interview “data,” and how best 
to ensure that research projects—based on interviews as the main method of 
data collection—were overall considered scientific. While not unproblem- 
atic, such insights have legitimized the use of interviews in social sciences. 

Informed by a “positivist” approach, such insights see interviews as infor- 
mative. Scholars, conceived as “minors” who unearth knowledge already 
there (Kavele and Brinkmann 2009), conduct interviews in order to dis- 
cover new factual data. Knowledge is conceived as given, and the role of the 
scholar consists in collecting objective evidence on social reality. Albaret and 
Deas tell us that “semi-structured interviews are often the only viable con- 
duit to discovering factual data, accessing certain types of information, and 
reconstructing a course of events or decision-making processes” (see chap- 
ter 5—Semistructured Interviews). Such assumptions have informed projects 
aimed at process-tracing events, processes, or “reveal” the points of views of 
certain groups of actors. When this is the case, scholars aim to access such 
views “as they are,” assuming that the discourses of those interviewed can be 
accessed in an objective way. 

As an effect of such assumptions, books on methods informed by this 
perspective have focused on the “interviewer's effect” (Hyman 1954) as 
well as the “reliability of the interviewee” and how best to control for these. 
“Validity,” from this perspective, is about asking the right questions, in the 
right way, and whether the interviewee offers truthful answers. 

In order to address the first concern (the “interviewer's effect”), exist- 
ing advice has focused on research design. The suggested recipe consists in 
focusing on the “scientificity” of the research methodology, which translates 
into the formulation of “unbiased” interview questions. Students have been 
advised to formulate “nonleading” questions that will not distort the inter- 
viewee’s responses (McCracken 1988). Questions should restrain, the word 
goes, from “putting words in the mouth” of interviewees. The emphasis has 
also been put on making interviewees feel at ease, in the hope that this 
would provide for a more honest conversation and thus access to “true” 
knowledge. While positivist approaches acknowledge that scholars who do 
interviews may not be “neutral” or objective, such bias is seen, however, as 
entirely quantifiable and controllable (Mosley 2013). 

In order to address the second concern (the “reliability of the inter- 
viewee”), existing insights point to the lack of neutrality of interview data, 
focusing on ways to “control” for this (for instance, Rathbun 2008). From 
this perspective the partiality of interviews is a problem. Insights have 
focused on memory failures, arguing for instance that “the older the witness, 
and the further from events they are, the less reliable the information” (Rich- 
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ards 1996: 200) or on interviewees’ deceptive strategies, when someone stra- 
tegically misremembers their accounts (Mosley 2013). In the context of IO 
research, this may occur when highly ranked bureaucrats for instance stick 
to an “official” discourse and do not reveal the politics behind the agendas 
of their organizations. Positivist scholarship on interviews here proposes tri- 
angulation as the golden solution, that is, cross-referencing across interviews 
conducted with different actors and between interviews and other forms of 
data. Davies for instance tells us that “the optimum solution appears to be 
a triangulation triad of primary sources (interviews, published first-hand 
accounts; and documentary sources (published or archival)), with published 
secondary-source information available” (Davies 2001: 78). Such combina- 
tion, the argument goes, makes for a powerful and rigorous research design 
(Richards 1996). 

Such questions, however, are relevant only when interviews are conceived 
as informative. When the value of interviews is seen as a manifestation of 
the interviewee’s views of social reality, the “lack of validity” of the inter- 
viewee account is no longer seen as a challenge to be overcome. Instead 
researchers can ask themselves new questions: how are such understandings 
produced, or what are the effects of such understandings? Rather than being 
concerned with a possible “misrepresentation” of reality by interviewees, 
their stories are seen as an exercise of sensemaking in a specific sociohistori- 
cal context, that is valuable as such. In IO research, the study of representa- 
tions has informed new agendas on the way IO bureaucrats look at specific 
governance problems or how such conceptions vary across IOs (Barnett and 
Finnemore 2004; Fresia 2009; Pouliot 2016; Beerli 2018). From this per- 
spective, “bias” does not need to be excluded and the interview text does 
not need to be “controlled” against other types of evidence. Triangulation 
becomes redundant. 

In the same way, interpretivists acknowledge that researchers need to make 
sense of their interview material. Interviews do not “speak for themselves” 
and need to be interpreted. This is unavoidable to the act of writing. Inter- 
views are, as put by Beerli in her discussion of biographical interviews in this 
volume (see chapter 6—Biographic Interviews), “dialogical,” “interactionist,” 
and “intrusive.” For interpretivists, the positionality of the researcher (what 
positivists call, in their own terminology, the “interviewer's effect”) affects 
the production of interview knowledge, as well as its interpretation. But of 
course the question then goes, how can we tell a good from a less convincing 
interpretation of an interview, if interview stories (and social reality more 
generally) can only be apprehended from the perspective of the researcher? 

One avenue consists in making the positionality of the researcher, as 
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well as the decisions made by the researcher during the research process 
itself, explicit. Leander for instance argues that “strong objectivity” consists 
in being reflexive about the researchers “presence in framing research” as 
well as the research process itself (Leander 2016). This implies discussing 
how our identity, stance, or emotions may affect our relationship to our 
research object as well as our research endeavor. A feminist researcher who 
examines IOs? institutional cultures will ask certain questions but also see 
certain asymmetries in gender relations that another researcher might miss. 
Not only the questions to the interviewees but also the questions to the 
interview text are affected by the positionality of the researcher. 

By the same token, researchers who are interpreting the interview text in 
light of a specific theory should discuss how their theoretical standpoint may 
illuminate their object but also close alternative interpretations (Kvale 1996). 
A marxist interpretation of international bureaucrats’ mindset will pay more 
attention to their economic vision of individuals and shared assumptions 
about the global economy more generally. This “lens” is not problematic as 
such, as it makes it possible for researchers to see things that might not be 
visible to others. It should, however, be made explicit. The first advice, thus, 
consists in making the researcher’s epistemological, theoretical, political, or 
emotional stance transparent. 

‘The second avenue consists in interpreting the interview text in light of 
the political, social, and historical context in which the text is produced. 
Paying attention to “context” may point for instance to the structural condi- 
tions that may constrain the interviewees when they speak. This mode of 
interpretation acknowledges that reality exists “beyond words” and recog- 
nizes the effects exerted by objective structures. Bourdieu, when discussing 
the interpretation of interview text, alerted us to the need “to understand the 
conditions of existence of which they (the interviewees and their words) are 
the product of” as well as the “the social effects which the relations of the 
fieldwork (. . .) can themselves exert” (1996: 30). In what he called a “realist 
construction” of interview text, “conversational analysis reads each discourse 
not solely in terms of its specific structure of interaction as a transaction, but 
also in terms of the invisible structures that organize it” (Bourdieu 1996: 
27). Prior knowledge of these objective structures is, thus, a condition for 
the interpretation of the interview text. 

These different contexts of interpretation, as described above, serve to 
make explicit the questions posed to the text. But this does not suffice. The 
interpretation of a text also includes requirements of credibility. Precise 
observations and logical argumentation are also necessary, even when inter- 
views and the scholarly work that interprets them are conceived as cocreated 
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knowledge. Several interpretations of interviews are possible but not any 
interpretation. 


Debates on Power Asymmetries 


Paying attention to the interview process and the relationship between the 
researcher and the interviewee therein calls for a reflection on power rela- 
tions, which are central in this relationship. A second controversy emerges 
when one considers this relationship from the perspective of power asym- 
metries. Most insights tell us that in the context of “elite interviews,” the 
power asymmetry plays in favor of the interviewee. Richards for instance 
tells us that “by the very nature of elite interviews, it is the interviewee who 
has the power” (Richards 1996: 201). Highly ranked officials can be difficult 
interlocutors who skillfully convey an official discourse and control both the 
format and the direction of the interview, as underlined by Dairon in this 
volume (see box j—Asymmetrical Interviews). The identity of the interviewer 
may also exacerbate this asymmetry: researchers who are young, female, or 
coming from an underprivileged social background can find themselves in a 
dominated position more frequently (Alles et al. 2016). 

But IO interviews can also be with nonelite officials, like local staff or 
beneficiaries. And mainly I would argue that the question of power, even 
in elite interviews, is not straightforward. When one places the interview 
relationship within its scholarly context, it becomes clear that the interview 
takes place for the purpose of the interviewer. As put by Kvale, the idea of an 
interview dialogue “gives an illusion of mutual interests in a conversation, 
which in actuality takes place for the purpose of just the one part—the inter- 
viewer” (Kvale 2006: 483). Given this, the idea of a genuine and even eman- 
cipatory interview dialogue is illusory. During the interview, the interviewer, 
who asks questions, sets the agenda. In light of this interaction, advice given 
to researchers on “how to gain trust” during interviews must be critically 
examined. We hear for instance that “the first question, crucial in initiating 
the exchange by gaining the respondent's trust, must not be too delicate or 
naive” (Alles et al. 2016: 118). But creating trust during the interview also 
serves the researcher’s purpose to “obtain a disclosure of the interview sub- 
jects’ world” (Kvale 2006: 482). Whether this is legitimate or manipulative 
is debatable. One may argue that elites, given their dominant social position 
or the—sometimes—contestable nature of their agendas or behavior, may 
well be the object of social scientific critique, and that such critique can be 
formulated only once elite discourses and practices are researched. But even 
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elites may be harmed—personally or in their career—by an interpretation 
of their thoughts or doings they fundamentally disagree with (Dauvin and 
Siméant 2001). This power asymmetry appears clearly when one considers 
what happens “after” the interview. The researcher indeed controls the inter- 
pretation of the interview text. Even when researchers adhere to an ideal of 
mutuality and coauthorship, the researcher organizes the process of knowl- 
edge cocreation. There is an ambiguity in the interview relationship that has 
both personal and instrumental elements. 

‘This brief discussion aims to encourage researchers to deeply reflect on 
the relationship between themselves, as researchers, and their interviewees. 
This relationship is, first, at the core of epistemological discussions on inter- 
view knowledge and its “validity.” Second, it is also at the core of a more 
ethical reflection on power asymmetries during the interview itself, as well 
as in the process of interpretation of interview text. 
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PART 3 


Documenting 


Introduction 
Documenting 


Fanny Badache, Leah R. Kimber, and Lucile Maertens 


International organizations (IOs) present themselves to the world through 
a large variety of productions and outputs. Reports and official websites are 
probably among the best-known objects through which IOs contribute to 
international politics. Yet IOs are also active producers of images, discourses, 
statistics, archives, and other artifacts. The contributions gathered in this 
section, entitled Documenting, are dedicated to methods applied to analyze 
IOs’ everyday productions. 

Disciplinary traditions and interests have often led scholars to specialize 
in the study of one specific type of production, developing tailored meth- 
odological devices to research them. For instance, traditionally legal docu- 
ments are studied by scholars of international law (see chapter 7—Legal 
Research), archives by historians (see chapter 8—Archives), maps by geogra- 
phers (see box m—Analyzing Maps). The contributions build on these disci- 
plinary developments to present well-established and proven methods while 
borrowing methodological tools from other disciplines. They show that dif- 
ferent scholarships bring fruitful and complementary insights to produce 
robust knowledge on IO documents. 

Indeed, the contributions propose methodological tools adapted to the 
nature of the productions but also adjusted to the context of IOs. While 
researching some products may need less methodological adaptation than 
others, contributors highlight what scholars should pay attention to when 
they study how IOs produce statistics (see chapter 12—Statistics and 
Quantification), when they delimit what an IO discourse is (see chapter 


113 


114 International Organizations and Research Methods 


11—Discourse Analysis), or when they work with IO archives (see chapter 
8—Archives). The contributions also attest to the increasingly diverse set of 
productions social scientists must make sense of. On the one hand, some 
objects have long been produced by IOs but have received less scientific 
scrutiny than others. For instance, while IOs have documented their activi- 
ties through visual objects, their creations, pictures, and videos have been less 
studied than treaties and resolutions (see box |— Visual Archives, and chapter 
9—Visual Methods). The same goes with budgets, which have received less 
attention than other written documents (see chapter 13—Budget Analysis). 
On the other hand, the contributions also address newly developed artifacts, 
especially in the field of digital communication: websites, tweets, data visu- 
alization, and so on. (see box p—Semiology of Websites, box q—Analyzing 
Tweets, and box s—Analyzing Charts, Infographics, and Dataviz). 
Documents and other objects offer a rich material to investigate IOs. 
‘The contributions present specific ways of studying IOs through their pro- 
ductions while raising questions of access and data management. Taken 
together, they show how the different methods help answer three main 
types of questions. First, the methods introduced in this section tackle ques- 
tions around IO identity, i.e., how IOs act out in their production, and 
IOs’ roles in world politics, i.e., what IOs produce and to what end. While 
the analysis of IO websites or annual reports (see chapter 10—Document 
Analysis: A Praxiographic Approach and box p—Semiology of Websites) pro- 
vide relevant data on how IOs present themselves, the study of statistics 
and maps shows how IOs contribute to governing the world by numbers 
(see chapter 12— Statistics and Quantification, box m—Analyzing Maps, and 
box s—Analyzing Charts, Infographics, and Dataviz). Second, exploring the 
elaboration, creation, and dissemination of IO productions is highly valu- 
able to understand the inner workings of these institutions. Indeed, looking 
at production processes enables the researcher to address questions around 
power relationships while rendering visible the dynamics of decision- 
making, categorization, selection, and framing. These methods help under- 
stand the complex functioning of IOs: by studying how a report is written 
(see chapter 10—Document Analysis: A Praxiographic Approach and chapter 
11—Discourse Analysis), an archive is sorted out (see chapter 8—Archives), 
a logo is designed (see box n—Branding Analysis), a colored badge is attrib- 
uted (see box o—Artifact Analysis), or a cover picture is selected (see chapter 
9—Visual Methods), scholars may capture the entanglements of hierarchi- 
cal dynamics, organizational cultures, professional habits, and individual 
decisions. Third, the contributions propose concrete methodological tools 
to analyze the content of IO productions and their broader effects. Schol- 
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ars can learn different techniques to interpret rich empirical material and 
study the social life of IO outputs, be it a specific concept elaborated in an 
IO report (see box r—Studying Ideas), a graph publicized on social medias 
(see box s—Analyzing Charts, Infographics, and Dataviz), or an artifact used 
in international negotiations (see box o—Artifact Analysis). The methods 
introduced in this part allow us to understand their explicit and implicit 
meaning(s)—what IOs say about the world through these productions and 
suggest ways to analyze their effects on global governance, like how IOs sup- 
port and sustain specific worldviews that impact the way global problems 
are governed (Interlude II— What IOs Talk about When They Talk about 
Themselves, and How They Do It). 

Some methods are more specific than others. Yet they often can be used 
to study a wide array of empirical material, like discourse analysis ready to 
investigate legal documents, policy reports, and advocacy pictures (see chap- 
ter 11—Discourse Analysis). Readers should therefore engage with different 
chapters and boxes to build their own methodological toolkit. For some 
productions, researchers may have to acquire specific technical competences, 
as in the study of IO maps. The contributions in this book provide concrete 
tools and advice to begin this journey. Overall, the increasing diversity of 
digital productions, only briefly addressed in this volume, builds a strong 
case for methodological dialogue, especially between qualitative and quan- 
titative methods. 


CHAPTER 7 


Legal Research 


Ian Hurd 


The structure and authority of international organizations (IOs) are 
expressed largely in legal language, and IO scholars should be comfortable 
with legal research methods. It is important to understand the basics of legal 
theory and treaty law as well as the dynamics of legal contestation. Insight 
on these topics is accessible via sociology, political science, history, and other 
fields in addition to legal studies; one need not be a legal scholar. Research 
on international institutions is enriched by seeing the legal system upon 
which global governance is based and the ways in which that system laces 
with international politics. 


What? 


IO scholars need to pay attention to legal issues because the organizations 
are constructed from legal resources, and their power, limits, and authority 
are defined by legal terms. The charters that establish IOs are legal docu- 
ments, the clauses that define their competence and obligations are read 
by lawyers, and the political disputes that ensue are fights over the who is 
allowed to do what under the rules. 

Law and legal argumentation are the common framework by which 
IOs intersect with government policies and their competing priorities. IO 
scholars should understand the law of treaties, models of legal change, and 
competing theories of interpretation that offer different views of the relation 
between texts and practices and between law and politics. 
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Most international institutions are anchored by interstate treaties that 
set out their purpose, structure, and powers. The Charter of the United 
Nations, the Rome Statute of the International Criminal Court, and the var- 
ious treaties that make up the European Union are examples of treaties that 
establish what used to be called public international unions and are now the 
international organizations at the heart of the system of global governance. 
These treaties are governed by international legal rules on treaties that define 
how they are made, interpreted, and contested. 

Legal research begins with reading the relevant legal texts. It then divides 
into distinct traditions. The first emphasizes the choices that actors make to 
comply with or to violate the rules. The second highlights the capacity of 
legal rules to constitute the social context, or the resources available to actors 
through the legal system. The first is roughly “positivist” in the social sciences 
sense and is suited to research questions that ask about patterns of compliance 
and violation. The second is “interpretivist” in the sense that it aims to under- 
stand how actors make sense of their choices and circumstances, with special 
emphasis on the productive power of legal forms in global governance. 

From human rights to financial flows to environmental collapse, govern- 
ment decisions are influenced by the rules, decisions, and powers of IOs: a 
government that wishes to subsidize a steel factory must consider how this fits 
with the rules of the World Trade Organization; Iceland’s decision to allow 
whale hunting is causing problems in its application to join the European 
Union; the International Court of Justice has some say over the legal implica- 
tions of Israel’s wall inside the Palestinian territories; the International Crimi- 
nal Court may have jurisdiction over US soldiers in Afghanistan despite the 
fact that the United States is not a legal party to the Rome Statute. 

Without attention to the international legal system that undergirds 
international institutions, one misses the fact that these political disputes are 
largely defined by legal terms, resources, and institutions. 


Why? 


International organizations rest on a foundation of law: they are established 
by legal devices—typically treaties—that establish a formal body with corpo- 
rate personhood and with powers that are defined by international law; the 
authority of international institutions is derived from international law, and 
so are their capacities, limits, and possibilities. Thinking about IOs requires 
paying attention to power, law, politics, and more all at once. If, as Martin 
says (2013), the effects of international law go far beyond the question of 
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whether a government complies with it, then research on how law relates to 
global affairs needs a broad lens on the productive power of law—that is, its 
capacity to generate new actors, new ways of engaging in politics, and new 
distributions of power. It also needs practical as well as theoretical analyses. 
To see the big picture of global affairs, we must first see the details. 

Carpenter provides an ideal example of the benefits of detailed legal 
analysis to illuminate the broad effects of international organizations. In 
her analysis of humanitarian rescue operations in the Bosnian wars (2000), 
she shows how the legal rules of the UN High Commissioner for Refu- 
gees (UNHCR) were exploited by belligerents with the effect that UNHCR 
humanitarian operations unwittingly served the Serb militias’ goal of ethnic 
cleansing. The UNHCR rules called for rescuing women and children from 
besieged Bosnian towns and leaving the men behind. This made it easier for 
the militias to empty the town and massacre the men. Carpenter lays out the 
legal framework in which the UNCHR operated in order to make sense of 
these tragic dynamics. 

Both the failures and the successes of IOs are shaped by the rules and 
competencies set out in law and from the interaction between the rules 
and the wider world of states and others who employ international legal 
arguments (including activists, individuals, and firms). It is as important to 
understand why these organizations sometimes fail to take collective action 
as it is to understand when they act. The international response to the Lib- 
yan uprising in 2011, for instance, was very different than to the Syrian 
uprising a year later, and the reason for this has much to do with the internal 
rules of the UN Security Council: the permanent members of the Council 
were much more divided over what to do about Syria than they were about 
Libya, and this put the Council in a very different state in relation to the 
two crises. Legal terms define the possibilities for politics, at the same time 
as political pressures shape the legal terms. 

The leading edge of scholarship on international law and IOs employs 
research methods that reveal the politicization of legal resources and the 
legalization of political struggles. Janina Dill’s study of the legal rules of war 
(2014) and Rebecca Saunders’ book on torture (2018) are excellent mod- 
els of research in settings that are characterized by a recursive relationship 
between the political power of law and a legal framing of politics. 


How? 


The choice of research method should be guided by the research ques- 
tion one is asking. On international law and international institutions, 


Legal Research 119 


four common question-types are “what is the law on this issue?,” “are 
governments following the law?,” “how did the law come to be?,” and 
“what follows from the way law and politics are constituted?” I address 
each briefly in turn. 

Scholars sometimes want to know what the law is on a particular 
subject—for instance, one might ask “under what circumstances can a 
UN peacekeeper use deadly force?” or “can the International Criminal 
Court investigate a crime by US soldiers in Afghanistan?” Answering 
these questions requires collecting the relevant legal texts (the UN Char- 
ter, Security Council resolutions, Status of Force agreements, for the 
first, and the ICC Statute and related material for the second) and com- 
paring their provisions with the situation at hand. These questions lend 
themselves to a kind of textualism—in the sense of starting with what’s 
written in the charter, treaty, court decision, or rule—but it can’t end 
there. With texts, two kinds of interpretation are inevitable, and they 
mean that competing accounts of the law are inescapable: interpreta- 
tion of the legal texts and of the empirical situation to determine how 
it relates to the legal frame. These are familiar as the main practice of 
courts: applying legal understandings to real-world circumstances. It is 
inevitable that disagreements will arise often over which documents are 
relevant and how they relate to each other and to the situation. For 
instance, US law forbids the ICC from investigating crimes committed 
by Americans while the ICC statute allows it if the crime was committed 
on the territory of an ICC member state. Scholars must be attentive to 
the possibility that the two may be impossible to reconcile. Questions 
about what the law is or says may end up with indeterminate answers, 
and yet the legal issue remains important. 

A related question is whether governments are complying with or violating 
the law. For instance, Hafner-Burton aims to document whether human 
rights compliance is improving over time or not (2008); Hathaway considers 
whether the United States violated international law when it killed Iranian 
general Qassem Soleimani in Iraq (2020). Making use of positivist social sci- 
ence methods, these studies begin by asking what the law permits or prohibits 
and then offer judgments about whether the state behavior in question falls 
within these terms. It is a popular approach among positivists in social sciences 
who seek correlations between features of the rules or the actors and the level 
of compliance (Koremenos 2016; von Stein 2013), but its outputs are only as 
good as the judgments it makes about what counts as compliance. 

“Compliance” is a tricky category given the high measures of ambigu- 
ity and contestation about what the rules are and how to interpret state 
behavior. In the face of fighting over whether behavior is law-abiding or not, 
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the positivist approach loses its footing. Brooks finds this fatal to efforts to 
determine whether American drone strikes are lawful or not. The wide range 
of legal instruments that are relevant to that determination leads to a great 
deal of ambiguity and contradiction in the answer. The legal sources include 
treaties, government statements and practices, and scholarly interpretations, 
which sometimes overlap each other and sometimes leave gaps that need to 
be filled by legal inferences from other issues. This complexity informs her 
conclusion that the drone program is profoundly destabilizing for the inter- 
national rule of law (2014). 

Alternately, one might ask about how and why the law came to be what 
it is. These historical questions are answered with reference to diplomatic 
history, often by reference to the travaux préparatoires—that is, the draft 
documents, negotiating history, and other supporting material that were 
produced in the course of the negotiating history of the treaty. Archival and 
historical research techniques are relevant here, but attention should also 
be paid to how the founding treaty has been changed over time through 
practice and reinterpretation. The legal charters of international organi- 
zations are like constitutional documents and their contemporary mean- 
ing may travel some distance from the original text. The UN Charter, for 
instance, says that resolutions of the Security Council pass only if they have 
affirmative votes from all five permanent members. However, in practice it 
has operated as if it said they pass as long as no permanent members vote 
against. This subtle change means that abstentions do not count as vetoes, 
motivated presumably by the interest of the permanent members in having 
a way to signal mild displeasure with a resolution while still allowing it to 
pass. This important innovation in the legal meaning of the Charter would 
be missed if one ignored practice and looked only at the formal text. 

Finally, one might ask what the legal system produces. This version of legal 
research addresses key issues of international law and politics, including why 
law matters, what it does, and most broadly “why should we care about inter- 
national law?” (Hakimi 2020). It is the interpretivist tradition in international 
legal studies. Legalization empowers some and disempowers others; it pro- 
duces new terrain for contestation; it changes the terms of political disputes 
and shifts the location and means of accountability (Shklar 1986). Peevers 
takes this approach in her study of international interventions and says it 
allows the scholar to see what law makes possible in addition to the standard 
question of what it forbids (2014). Recognizing that the legal system as pro- 
ductive and empowering for some in addition to being regulative and con- 
straining (Hussin 2014) is the key to opening a broad and grounded empirical 
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research program on the implications of how law and politics are constituted 
and defined, sometimes as one and sometimes in distinction to each other. 


What Challenges? 


The tools of legal research described above are available to scholars from 
all disciplines, but there can be some danger in talking across disciplinary 
boundaries because the normative assumptions of legal scholarship often 
differ from those of social sciences. Scholars should be alert to the risk of 
misunderstandings when these different normative lenses are not acknowl- 
edged. Specifically, IO research needs to guard against the assumption that 
international law and organizations are naturally good and progressive in 
world politics. This premise is common in legal scholarship despite its obvi- 
ous empirical and political problems. And while it can impede empirical 
scholarship, luckily it is easy to avoid. 

It is common for scholars to make impassioned pleas that international 
law and institutions should be more empowered and more respected. 
Hathaway is confident that “international law is frequently the simplest, 
most effective and least expensive solution to problems big and small” 
(2007: 3). Slaughter believes that international law indicates the correct 
policies for governments, on the assumption that law represents “multilat- 
erally by agreed-upon rules of the game” with benefits for all (2013: 616). 
O’Connell goes further: she chides scholars who find fault with interna- 
tional law because “given the nature of the problems we face in the world, 
(. ..) any effort to weaken international law only serves to undermine the 
prospects for achieving an orderly world and progress toward fulfillment of 
humanity’s shared goals, including prosperity” (2008: 14). These scholars 
embody the view that international law and institutions are enchanted 
spaces where political disagreements disappear and are replaced by good- 
for-everyone outcomes. 

‘This enchanted view of international law and institutions derives from 
two strands of liberal theory in international relations that sees international 
institutions as manifestations of interstate cooperation and as inherently 
beneficial, naturally apolitical, and worthy of expanding. 

Legal scholarship on IOs should take a more empirical and less ideologi- 
cal approach, attentive to the costs as well as the benefits of law. Since people 
have different interests and desires it is inevitable that international legal 
institutions will favor some interests rather than others. It is inconceivable 
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that international law could make all the people happy all the time. Scholars 
should look for the moments when trade-offs among competing interests 
are occurring. Indeed, the political controversies around global governance 
are evidence that people and states are deeply invested in how institutions 
redistribute goods and harms. Scholars can avoid a priori normative com- 
mitment to the desirability of international law and institutions and instead 
begin with a neutral position that looks first at the empirical evidence and 
only later draws political or normative conclusions. 


To Go Further 
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CHAPTER 8 


Archives 


Ellen J. Ravndal 


International organizations (IOs) have left behind an abundance of docu- 
ments and other archival material. Combined with state and personal col- 
lections, these offer a rich basis for IO studies. Researchers can use archives 
to answer questions of institutional history—the internal developments of 
IOs or IOs’ role as international actors—and to examine broader processes 
of international history—seeing IOs as an arena for and mirror of global 
developments. 


What? 


Archives are institutional repositories of documents. Archives can be large, 
professional institutions, or smaller and more ad hoc. They can be operated 
by national or regional governments, IOs, nongovernmental organizations 
(NGOs) or other private associations, or hosted by libraries or museums. 
When we talk about “the archive” we most often refer to a physical location 
where documents are stored and where researchers can consult material in 
reading rooms. However, new technology now means that some documents 
can be accessed online, either through archives’ own websites or in digital 
collections or databases. This chapter primarily discusses traditional, physi- 
cal archives, where the majority of documents still reside, but similar con- 
siderations apply to research using digital archival collections. Archives are 
important because they contain “eyewitness accounts,” or “primary sources,” 
and unpublished information that cannot be found elsewhere. 

Archival research is an old method. Scholars first developed tools and 
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procedures for working with archival collections during the sixteenth and 
seventeenth centuries (Arnold 2000: 45). Leopold von Ranke (1795-1886) 
is commonly presented as the father of modern history because of his empha- 
sis on using primary sources. Ranke argued that historians should produce 
“scientific” and “objective” history based on “evidence.” However, archival 
research is not quite as straightforward as letting the documents speak for 
themselves. Sources, including archival documents, can only be understood 
through the researcher’s interpretation. They are a “residue” left behind by 
past human activity (Bloch 1992 [1954]: 45) and represent “structures that 
survive into the present” (Gaddis 2002: 35). Therefore researchers cannot 
simply enter the archive to find a perfect picture of past events and processes. 
(For more on analysis of documents, see chapter 10—Document Analysis: A 
Praxiographic Approach.) 

IO research can include sources from many different archives. IOs them- 
selves often have rich archival collections that can shed light on the IO’s 
internal institutional developments (e.g., Gram-Skjoldager and Ikonomou 
2019) or provide a picture of broader societal developments (e.g., Ravndal 
2020). Various national archives hold material sent home from state delega- 
tions to IOs. Finally, private individuals who worked for IOs, or NGOs and 
civil society groups, may have left collections in university libraries, national 
archives, or the archives of political parties, or similar. Tracking down such 
private collections can be challenging but rewarding. 

A growing IO literature uses archival research. Two studies taking the 
League of Nations as their starting point represent excellent examples of how 
to conduct archival research on IOs: Patricia Clavin (2013) examined how 
the League expanded its work on economic and financial policy and how 
this laid the foundations for the post-World War II Bretton Woods system, 
while Susan Pedersen (2015) analyzed how the League’s mandates system 
sought to protect the interests of imperialism while inadvertently opening 
up space for anticolonialist actors and arguments. 


Why? 


Archival IO research brings many benefits that stem from the very defini- 
tion of what an archive is: a repository of unpublished “eyewitness accounts” 
produced by the day-to-day operations of the organization. This can include 
many types of documentation, such as incoming and outgoing telegrams 
and letters commenting on the events of the day, various memoranda with 
contemporary analysis of events or proposals for new policies, drafts and 
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redrafts of speeches that can provide insight into the balancing between dif- 
ferent priorities, or verbatim records of meetings and marginal notes and 
comments that can show internal disagreements or emphasize what different 
individuals considered important. Contrary to the polished and carefully 
crafted words of published press statements and reports, these documents 
offer a glimpse into the “behind-the-scenes” operations of the organization. 

The benefits of consulting these materials are obvious. First, archival 
sources provide the possibility of constructing a narrative of events as seen 
by IO staff and offer a chance to grasp the spur of the moment, instinctive, 
and “true” reactions and thoughts of the actors working in these organiza- 
tions. Second, working with material from different archives allow research- 
ers to see an event from multiple vantage points. Indeed, historians turned 
to IO archives in the past two decades precisely to counteract the national 
bias of traditional diplomatic history (Amrith and Sluga 2008; Kott 2011). 
‘The most fruitful archival research projects will be those that combine mate- 
rial from IO archives, national archives, and private collections, because this 
allows for multiple perspectives on the same phenomenon. A third benefit 
is that archives contain unpublished information, allowing the researcher to 
bring new insights into the academic debate. Finally, archival material can 
be analyzed with different research methods that offer grounds for method- 
ological innovation and interdisciplinary dialogue. 


How? 


As Marc Trachtenberg notes, although many believe archival research is 
“daunting,” there is really “nothing mysterious about working in archives. 
No arcane set of skills is needed” (Trachtenberg 2006: 165-66). It is largely 
a research method based on common sense that most scholars learn through 
practice and by asking advice from experienced researchers. This section 
sums up the lessons I have learned from visits to multiple archives in the 
course of my research on IOs (e.g., Ravndal 2020; 2023). 

For clarity, it can be useful to separate the research process into three 
stages: before, during, and after the visit to the (physical) archive. Similar 
questions arise when working with digital collections. 

‘The first stage involves identifying archival collections of interest and 
how to access them. A common way to find relevant collections is through 
the citations in publications on the topic. Additionally, historians place great 
value on locating novel or hitherto underutilized sources. For IO research, 
the logical starting point is to check whether the organization’s own archive 
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is open to researchers. Most large IOs such as the UN, the International 
Labour Organization (ILO), or the Food and Agricultural Organization 
(FAO) have archives, usually located at their headquarters. I have often 
found that a visit to the UN archive in New York or the League of Nations 
archive at the UN Library in Geneva is useful even if these organizations 
are not the main object of study. Since their foundation, the UN and the 
League functioned as hubs for international cooperation, and we can there- 
fore find traces of other IOs and broader international developments in 
their archival collections. Likewise, many researchers routinely consult the 
national archives of the United Kingdom and/or the United States because 
their extensive diplomatic network collected information about events all 
over the world, and their archives are relatively easy to access and navigate. 

Once interesting archival collections have been identified, accessing 
them usually involves travelling to the location of the archive and spend- 
ing at least two to three days there. Such visits must be carefully planned. 
Archives’ websites give information about opening hours, access applica- 
tions, and whether it is necessary to book a seat in their reading room. Addi- 
tionally, it is a good idea to consult finding aids, and seek advice from the 
archivists and/or scholars who have visited before, to identify (some of) the 
specific documents you would like to view. Many archives allow for ordering 
documents in advance to facilitate more time-efficient visits. 

To locate relevant information in the archives, it is necessary to under- 
stand how documents have been created and organized. Archives often have 
online catalogues that allow keyword searches, but their file descriptions are 
of varied quality. Relying on an online search risks missing out on relevant 
files if the archivist who catalogued them failed to include “your” keyword. 
Instead try to understand how the organization produced and catalogued 
its documents. Organizations may use a central registry system, thematic 
codes, a chronological system, or a system centered on different departments 
or individuals. I have found relevant information, for example, by going 
though folders of all incoming and outgoing telegrams for an office dur- 
ing a given time period. None of these telegrams would have come up in a 
keyword search. 

‘The second stage of archival research is the actual visit to the archive, for 
which my main advice is: bring a camera and take good notes. Most archives 
today let visitors make digital copies of documents with a camera or phone. 
But not all, so do read the instructions on the website carefully beforehand. 
Because international travel is expensive, researchers may only be able to 
spend a few days at the archive, but by photographing documents even a 
short visit could yield thousands of copies that can later be analyzed at a 
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slower pace. Keep in mind that each day in the archive will likely correspond 
to a week or more sorting through and analyzing the copied documents. 

It is very important to take good, informative, and organized notes 
during archival visits, so that the researcher later knows which file a docu- 
ment belongs to and how to locate information in the photos. A note or 
photo without its document reference is useless as it cannot be included 
in the analysis. Software like RefWorks, OneNote, or NVivo can be use- 
ful. My own system is quite low-tech. I create a word document titled with 
the reference to a folder or box or other small archival unit (e.g., “FO 371 
59719”) and sort these into computer folders that replicate the structure of 
the archive. In each word document I take (brief) notes about what I found 
interesting about this unit and which parts I have photographed. When 
taking photos, I start each sequence by photographing the archive box, the 
front of the folder, and/or the call slip—some visually striking element that 
allows me to quickly identify the start of a new unit in my photo stream, 
and which will later provide information about the document reference for 
the following photos. 

Finally, the third stage of archival research involves sorting through and 
analyzing the documents and integrating the information in your writing. 
Archival collections have different copyright policies. To publish (excerpts 
from) a document, or even just include a quote from it, it may be necessary 
to get permission from the copyright holders. Consult the copyright infor- 
mation on the archive website or ask an archivist. 

There are different ways to cite information from archives in publica- 
tions. Sometimes the archive gives a suggestion, or a journal may prefer one 
style over another. The most important consideration is that it should be 
possible for others to locate the exact document. Therefore each reference 
must include three elements: 1) something to identify the individual docu- 
ment (e.g., “Letter from Smith to Johnson, 12 October 1956” or “Min- 
utes of meeting, 3 January 1948”), 2) the archive reference for the relevant 
folder or box (e.g., “S-0194-0001-02”), and 3) the name and location of the 
archive. 


What Challenges? 


Working with archives presents both practical and methodological chal- 
lenges. Visits to archives can be both expensive and time consuming. Some 
scholars may face additional challenges with visa applications, not to men- 
tion the obstacles to international travel demonstrated by the COVID-19 
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pandemic. This means that traditional archival research may not be viable 
for everyone or at all times. 

Some archives are taking steps to remedy these problems by digitizing 
their collections. The UN Library in Geneva, for example, now offers online 
access to the entire League of Nations archival collection.’ The International 
Telecommunication Union (ITU) is also making parts of the ITU’s docu- 
mentation available online. Several other archives have digitized parts of 
their collections, and these are mostly free to download through their online 
search catalogues. For a useful overview of relevant online archives for IO 
research, see the website of the United Nations History Project.’ 

Methodological challenges also arise due to how archival collections have 
been produced, preserved, and organized. Not all sources survive into the 
present, and importantly, “sources of the past do not survive in neat patterns 
of their own accord” (Arnold 2000: 60). Therefore archival collections will 
only ever give a partial picture of the past, and it will be a picture that is 
“biased” in significant ways. 

Bias in a collection can arise from different sources. Sometimes what 
makes it into an archival collection can be accidental. Fires and floods have 
been and remain a hazard for archival collections. But documents are at risk 
of more than natural disasters, wars, and revolutions. As researchers we rely 
on the work of archivists in organizing, managing, and preserving the collec- 
tion. Archives have been produced for a purpose, and this purpose is not pri- 
marily to serve academic researchers. Past archivists have made choices about 
what to keep and what to destroy, and these processes can leave systematic 
biases in the collection. The researcher should therefore pay attention not 
only to what they find in the archives but also to what is not there. “Opera- 
tion Legacy” represents a large-scale example whereby the British govern- 
ment during decolonization destroyed or withheld numerous documents 
from its colonial administration (Sato 2017). On a smaller scale, the archival 
collection of Trygve Lie, the first UN secretary-general, contains folders and 
documents that he organized when writing his memoirs in the 1950s. Can 
we be sure he did not seek to present himself in a good light by leaving out 
certain files? To deal with biased sources the researcher must first identify 
how the source is biased, and second seek to compensate by comparing, or 
triangulating, information from different sources. 

Whether intentional or accidental, the gaps and silences in archival col- 
lections are significant, and the researcher must be aware of them. Criti- 


1. https://archives.ungeneva.org/lontad (accessed February 9, 2023). 
2. https://www.histecon.magd.cam.ac.uk/unhist/index.html (accessed June 7, 2022). 
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cal, feminist, and postcolonial scholars are particularly attentive to these 
silences and have developed methodologies for dealing with them (see box 
x—Postcolonial Insights and chapter 28—Feminist Approaches). Lisa Lowe 
(2015: 5) discusses the need to “read across the separate repositories orga- 
nized by office, task, and function, and by period and area” when working 
with the archives of the colonial state. In IO research, scholars can include 
archival collections in the Global South as an additional perspective to avoid 
replicating the worldview found in the American and British collections 
many of us continue to work with. Recently published work shows the value 
of including archival visits to countries such as Ghana, India, and Trinidad 
(O’Malley 2018; Getachew 2019). 


To Go Further 


Clavin, Patricia. 2013. Securing the World Economy: The Reinvention of the League of 
Nations, 1920-1946. Oxford: Oxford University Press. 

Pedersen, Susan. 2015. The Guardians: The League of Nations and the Crisis of Empire. 
Oxford: Oxford University Press. 

Trachtenberg, Marc. 2006. The Craft of International History: A Guide to Method. 
Princeton: Princeton University Press. 

The United Nations History Project. 2021. https://www.histecon.magd.cam.ac.uk/un 
hist/index.html (accessed June 7, 2022). 
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BOX L 


Visual Archives 


Valérie Gorin 


Since the 1860s—1870s, drawings, paintings, posters, daguerreotypes, post- 
cards, stamps, pictures, films but also leaflets and periodicals are preserved 
by the International Committee of the Red Cross (ICRC)! or the Inter- 
national Federation of the Red Crescent (IFRC),” and also by the United 
Nations where each agency has its own visual archives (such as the United 
Nations Refugee Agency or the World Health Organization). Exploration of 
visual archives can enrich or fill gaps, as well as open new lines of enquiry 
over the imperialist and colonialist ideologies of the late nineteenth century, 
or the settings of multilateral diplomacy in the interwar period of North- 
South relations after 1945. For example, a historical investigation over the 
poster archives of the International Red Cross and Red Crescent Museum— 
the largest humanitarian collection in the world—has questioned the visual 
politics and the reappropriation of humanitarian principles throughout the 
Red Cross movement since the First World War (Gorin 2019). 

Most of the time, visual artifacts are preserved in separate archival units, 
collections, or buildings, even in the same organization, depending on the 
visual or textual nature of the source.’ The constellation of archival sites 
thus represents multiple challenges for the researcher. It is greatly advised to 
contact the archivist in charge of the visual archives in the organization (see 
chapter 8—Archives). Some archives are digitized online for free, but they 


1. The ICRC audiovisual archives. https://avarchives.icrc.org/ (accessed October 2, 2020). 

2. The IFRC historic film collection is on Youtube: https://www.youtube.com/channel 
/UCd2bE77hqagP0uP2JxfSTdA (accessed October 2, 2020). 

3. For example, see the distinction made in the WHO archives: https://www.who.int/arch 
ives/fonds_collections/partners/en/ (accessed October 2, 2020). 
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do not include all visual artifacts (e.g., memorabilia). An on-site visit to the 
archives is necessary for researchers interested in two aspects: first, a focus 
on the materiality of visual objects, such as the technique or the constraints 
related to viewing devices, will allow researchers to understand the immer- 
sive experience of spectators from previous decades; second, the consultation 
of written material accompanying images is mandatory (e.g., reports, per- 
sonal correspondence, field notes) to have contextual information about the 
production/dissemination process (see chapter 9— Visual Methods). Visual 
archives preserved in the same location offer homogeneous corpuses that 
give meaningful inputs about the use and reuse of an image, sometimes for 
multiple purposes and through different visual formats. It is therefore useful 
to explore the political and cultural specificities of images and their varia- 
tions through space and time. However, it is not always possible to trace the 
history of their creation. Many images before 1945 are preserved without 
any captions, nor location, date, or name of the image-maker. Finally, acces- 
sibility is also an issue: images of the early twentieth century are accessible 
because they fall into public domain. However, there might be a limitation 
period of approximately twenty to forty years on contemporary images for 
confidentiality reasons. 
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CHAPTER 9 


Visual Methods 


Valérie Gorin 


When you want to observe the organizational structure of IOs, as well as 
understand their history and culture, visual methods offer highly valu- 
able approaches. Producing images about or analyzing images of IOs allow 
researchers examination of their representations of the social world, their 
visual conventions, and their power dynamics. Here we introduce several 
case studies using visual data as sources of information, and how to proceed 
with image-based research. 


What? 


Rooted in social sciences, visual methods sit at the intersection of two dif- 
ferent paradigms: “[they] are research practices that explicitly use images 
in various ways including drawing, photography, video, film, and Internet 
pages. The images are either regarded as a source of data in themselves, or 
as a way of producing data through their use, or a combination of the two” 
(Warren 2009: 566). Researchers who work on images as sources of data 
act as “image collectors”: they analyze the significance and meaning of pre- 
existing images, whether produced by an institution or by previous studies. 
Their methods include the use of visual archives and databases, visual arti- 
facts, websites, institutional documents, or the study of the architecture of 
buildings and places. On the contrary, researchers who produce visual data 
are “image generators”: they relate to images explicitly generated for their 
study, whether by the researchers or in collaboration with research partici- 
pants. Rather than using words, these methods include direct observation 
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(of behaviors, settings, rituals, etc.) but also visual recording of what is going 
on (drawings, paintings, photography, film, cartography, etc.). 

Interestingly, most authors agree that visual methods are often ignored 
in organizational research, despite a growing interest since the early 2000s 
(Ray and Smith 2012). To date, only one handbook promotes the useful- 
ness of visual methodologies in organizations (Bell, Warren, and Schroeder 
2013); however, visual theory and culture has been a flourishing field of 
study, especially in sociology and anthropology. For example, visual meth- 
ods allowed researchers to explore the creation and dissemination of visual 
clichés among IOs—as suggested by the previous work of the author about 
humanitarian imagery (Gorin 2012). In addition, participatory visual meth- 
ods have increased recently, explicitly involving “research respondents in the 
co-creation of qualitative data’ (Vince and Warren 2012: 275). They use 
either photo-interviews, also called photo-elicitation (production of pictures 
by the researcher and interviews with participants to elicit discussions about 
what they see) or photo-voice (participants make pictures themselves, often 
to reveal their preoccupations or agendas). 


Why? 


We live in an image-saturated world, where image production, manage- 
ment, circulation, and consumption have pervaded every organization in 
the private and public sector. Therefore visual methods enable more in-depth 
overviews of the different dimensions of organizational settings, including 
the symbolic, cultural, or emotional. They help to question the way IOs 
perceive themselves and build their identity and reputation, both in their 
visual branding or artifacts (logos, merchandising, uniforms, etc.) and their 
communication strategies (such as accountability, field evidence, fundrais- 
ing, awareness or advocacy campaigns, and forensic data). Using visuality 
allows better observation of the organizational structure but also intra- and 
interorganizational interactions, emphasizing the power dynamics among 
them, which sometimes remain invisible even to those who work in these 
organizations. 

Overall, images are considered reliable because of their so-called unbiased 
transparency. However, as in any qualitative approach, images must be con- 
sidered subjective expressions of points of view that refer to cultural practices 
and cognitive resources that determine the way we see the world around us 
(Berger 1972). Visual methods allow the exploration of representations of 
power, gender, race, or ideology at work among IOs. For example, there is 
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a growing scholarship on visual histories of IOs, especially those active in 
the humanitarian sector. Research has shown how UNHCR images shaped 
national perceptions of and international policies about refugees, or how 
WHO visual campaigns contributed to promote health standards (Gatrell 
2011; Rodogno and David 2015). In our own research, we focused on the 
circulation of colonialist imagery in humanitarian organizations active in 
Africa since the nineteenth century, despite their claim for dignity and uni- 
versalism (de Laat and Gorin 2016). To reduce the variety of visual materials 
available, we focused on photographic archives—pictures being one of the 
first mass media introduced by the use of mechanical reproduction. 


How? 
Visual Corpus 


‘The constitution and size of the corpus matter. First, determine the purpose 
of your research and your selected approach in visual culture. If you are 
interested in an iconic image (like a logo) or a series of posters for a given 
campaign, then it will be important to focus on the description and detailed 
analysis of each image. On the contrary, if you aim to identify visual conven- 
tions and the emergence of new representations over space and time, then 
it is better to select a large visual corpus, so you can compare similarities 
and differences between images. Dealing with a visual macro-corpus can 
be impractical, so it can be useful to rely on content analysis softwares such 
as Atlas.ti that allows you to process through description and classification 
devices. One recommendation would be to start with a smaller corpus on a 
short period (e.g., a decade), while mixing sources among IOs to identify if 
there are any recurrent or new visual conventions. 

Second, think about your capacity to apply visual skills. Working with 
one or several visual methods implies knowledge about visual techniques 
(use of the medium), visual theories (semiotics, iconology, etc.), and visual 
styles (such as medical or travel photography). Gillian Rose’s handbook will 
help you become familiar with visual methodologies, as well as provide a 
list of questions to interpret images (Rose 2016; Cohen and Ramel 2018). 
Whether you are an image collector or an image generator, you need to fol- 
low several steps. Beginners in visual methodologies may be interested in 
following Luc Pauwels’s step-by-step integrated framework for visual social 
research (2011). 
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Images as Data 


If you plan to analyze images, the first step aims at identifying institutional 
archives, either physical or digital, by establishing a list of major IOs related 
to your topic (see also box |—Véisual Archives). To find out visual databases, 
you have three options: whether the organization has a library or documen- 
tation center, archive units, or a media center. Many visual databases are 
either unrecognized or underused by researchers interested in international 
relations—such as the UNHCR! or the ICRC.” Many of them are open 
to the public, but if not you should seek whether there is an exception for 
scholars. In many cases, a visit to the physical archives or library in the head- 
quarters could be useful. Although using online research engines or data- 
bases is helpful, sometimes they do not include everything. Therefore it is 
also rewarding to contact librarians working in these organizations,’ so they 
can help you identify specific collections and get familiar with the databases. 
The secondary literature can also be useful (especially in history, ethnogra- 
phy, political sciences, or international relations) to help locate visual mate- 
rials previously used or mentioned by researchers. Make sure you find visual 
databases with sufficient information about image producers (e.g., personal 
biographies, mandate framework with the IO),* the context of images, their 
dissemination, and possibly the targeted public. 


Images That Produce Data 


If you plan to generate images, you have to decide whether you want to pro- 
duce images yourself or if you need to include research participants within the 
organization. Many layers of organizational settings can be directly observed 
and visually recorded: employees, buildings, meetings, interactions, processes. 
Try to remain unobtrusive; this can happen with photographic devices or 


1. The UNHCR visual media center can be found at: https://media.unhcr.org/C.aspx? VP 
3=CMS3&VF=Home (accessed October 2, 2020). 

2. In 2015, the ICRC put all its audiovisual collection online: https://avarchives.icrc.org/ 
(accessed October 2, 2020). 

3. Most of the time, you find a contact email on the archive resources or library webpage 
of the organization. 

4, For example, in 1948 UNICEF partnered for the first time with David “Chim” Sey- 
mour from the newly founded Magnum agency, to attract international attention and funds 
for war orphans by using one of the best names in war photography. 
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videotaped recordings, so you might opt for drawings, diagrams, or graphs 
instead (Meyer 1991). Conversely, you can include participants to make 
images of places that would otherwise remain inaccessible to the researcher. 
Thus Samantha Warren asked members of an IT firm to picture their desks to 
explore the aesthetics of work environments (Warren 2002). This option will 
include field preparation with the IOs and instructions to participants. This 
is a very important aspect: you need to have a mutual discussion over the set- 
up and execution of the study. Some legal and ethical issues have to be agreed 
upon; for example, you need to know whether subjects will be aware they 
are being recorded and whether they agree (informed consent). Many IOs 
have working agreements already established with photographers that might 
be useful and readjusted by researchers, whether to obtain consent or to pay 
attention to ethical issues when using or producing images.’ 


Visual Analysis 


‘The second step of your research will consist of analyzing visual products 
based on what is represented (the content) and how it is represented (the 
visual style). This implies you should pay attention to the social life of the 
image: its production, content, circulation, and reception. The circumstan- 
tial elements are important to understand why an image was made, what 
the communication objectives of its mediation were, and when and where 
it was disseminated and how often. Never separate the image from the text. 
In her analysis of the UNHCR and other refugee libraries’ pictures, Caro- 
line Lenette thus offers a step-by-step iconographic approach to describing, 
analyzing, and interpreting the structure, contexts, and producers of refugee 
photography throughout the 1980s—2010s (2016). An image is always the 
result of a choice; it can have several meanings depending on the political, 
social, economic, or cultural contexts in which it is created and seen. In our 
study of the iconology in Africa, we compared large corpuses of pictures 
with former studies on iconographic conventions in the humanitarian sec- 
tor, to explore their symbolic dimension (exoticism, racial gaze, Western 
governance) throughout the twentieth century. The persistence of clichés 
through space and time and through different visual products (posters, post- 
cards, stamps, films) can help categorize representations between typical 
(recurrent, generic) and particular (unusual, exceptional), for example. 


5. See for example the photograph consent form from UNAIDS: https://www.unodc.org 
/pdf/india/news_and_events/photographer_guidelines.pdf (accessed October 2, 2020). 
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If visual data were generated to include the “voices” of the stakeholders, 
the researcher has to question the encoding-decoding process. This means to 
plan qualitative interviews with the research participants, so they can explain 
their views, their choices, what they take for granted. It allows the researcher 
to avoid assumptions, bias, or question elements that might not be obvious 
or relevant to him/her.° 


What Challenges? 


Using visual methods in IOs involves two main challenges: the political con- 
text of such organizations and any ethical issues. IOs are institutions where 
political stakes, behind-the-scenes meetings, and multilateral diplomacy 
play a crucial role. Photographers working in these organizations usually 
have to follow strict media accreditation and image management. First, the 
researcher who wishes to use direct observation or visual recordings should 
expect some resistance to setting up the methodological framework, for they 
would not be allowed to access all places freely, especially those involving 
sensitive political issues or private meetings. Second, it might be difficult 
to identify appropriate stakeholders in IOs (either the staff, the decision- 
makers, or any state representative) and have access to them or engage them 
in the study. This is even more obvious when using participatory visual 
methods in the context of IOs working with affected populations, such as 
the UN agencies. Be flexible and realize that sometimes it is better not to 
take the picture. 

Ethical issues, on their side, relate to anonymity (e.g., blurring faces), 
intrusiveness (sensitive cases or places, disturbance of the interaction), and 
confidentiality (respect of privacy, protection of vulnerable people). In the 
case of visual participatory methods, specific ethical principles related to the 
display of suffering, including dignity and protection of the subject, must 
be considered before any visual experiment proceeds, as the research should 
avoid the perpetuation of visual clichés. The researcher working on images 
as sources of data should also discuss ethical distancing from stereotypes. But 
in this case the challenges relate to the copyright and the control and use of 
public and private images. Many IOs mandate external, public, or private 
image producers (e.g., filmmakers, photographers, and publicity agencies) 
to produce visual artifacts. Therefore you might struggle to obtain the rights 


6. See a brief given to research participants by Vince and Warren (Vince and Warren 2012: 
295). 
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to use/reproduce them in a published work. Questions over ethical issues, 
authorship, and ownership of images should be anticipated, discussed, and 
negotiated previously in an agreement with the IO (Kunter and Bell 2006: 
186-90). 


To Go Further 


Margolis, Eric, and Luc Pauwels, eds. 2011. Zhe SAGE Handbook of Visual Research 
Methods. London: Sage. 

Pink, Sarah. 2001. Doing Visual Ethnography: Images, Media and Representation in 
Research. London: Sage. 

Rose, Gillian. 2007. Visual Methodologies. An Introduction to the Interpretation of Visual 
Methods. London: Sage. 

Van Leeuwen, Theo, and Carey Jewitt, eds. 2001. Handbook of Visual Analysis. Lon- 
don: Sage. 
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BOX M 


Analyzing Maps 


Benoit Martin 


We consider a “map” to be any picture showing information on the repre- 
sentation of a geographical space. Whether they deal with networks or terri- 
tories, maps have long been a device of international relations used to move 
around, census lands, delimit sovereignties, divide regions, plan attacks, glo- 
rify empires, iconize nation, and so forth. 


lOs as Cartographers 


Despite the diversity of actors who produce maps, IOs remain essential pro- 
ducers at the international level. Their maps fulfill two main functions: 


Field operation support when accuracy is required (location, timeli- 
ness, data attributes). Maps can be complex, mixing geographical 
information systems, surveying, modeling, remote sensing, big 
data, and so on. UNDPO, UNHCR, UNEP, or IOM produce 
such maps. 

Analytical support for argumentation (synthesis of global trend, illus- 
tration of case study). Those “thematic” maps, as well as tables 
or dataviz, often enrich analytical publications (reports, notes, 
manuals). 


Both uses present technical insights to convey indisputable evidence. 
Such aura constitutes an obstacle to analyze maps that must be overcome. 
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Maps as Discourses 


Cartographic language is a specific type of discourse with its own set of codes 
and methods (Bertin 1967). These best practices, which some specialists like 
to consider rules, do not prevent subjectivity. Drawing a map consists of a 
succession of choices, which are not automatic or neutral. For example, authors 
select a base map, with its own distortions, centering, and boundaries, or 
pick colors, with visual weights, cultural references, and even emotional sig- 
nificances. Maps are not disembodied objects but closely depend on their 
production context and target (Harley 1989; Wood 1992) with rhetorical 
dimensions and sometimes an instrumental purpose. 

In the UN context, the Geospatial Information Section, responsible for 
clearance process of any UN map, sets standards (graphic and toponymical) 
about accepted international borders. UN maps thus reflect power dynam- 
ics and issues among its members through footnotes on territorial disputes, 
dotted lines for undetermined boundaries, or even continuous lines inside 
sovereign states.' Otherwise, the notorious reports of the Intergovernmental 
Panel on Climate Change skillfully maps a borderless Earth in order to alert 
the public about the global dimension of the planet’s warming, while at the 
same time gridding the world according to countries when states have to 
take actions. Maps mostly serve as medium for adopted models on future 
possible climate scenarios. 


Apply a Critical Look on IO Maps 


Maps are a specific kind of dataviz and present the same difficulties in their 
analysis. It must consider the shape, the content, and the two combined (see 
box s—Analyzing charts, Infographics and Dataviz). The triple questioning— 
what does the picture show? who produced it/for what purpose? how was it 
built?—is fundamental for analyzing maps. 

Considering that a “good map” reflects intentions of the author/sponsor, 
such pictures lead to three main effects for international relations: 


1. For example, the Indian State of Arunachal Pradesh is isolated from the rest of the 
country by a continuous line, the same as for international borders, because of Chinese claims 
of this area. 
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BOX N 


Branding Analysis 


Stefan Tschauko 


Branding and brand management, typically associated with marketing in 
the for-profit sector, is increasingly being adopted by international organi- 
zations (IOs). IOs are now creating, managing, and implementing holis- 
tic branding strategies. These encompass visual identities, carefully selected 
names for specific campaigns, and engagement with stakeholders through 
curated brand touch points, such as social media channels. 

Branding has been studied in many contexts, including corporations 
and nonprofit organizations (Kapferer 2012; Laidler-Kylander and Stenzel 
2014). Research on IO branding, however, is scant despite the potential ben- 
efits that can be derived from IO communication practices and strategies. 

‘Three types of questions can guide research on branding in IOs: 1) What 
is branding in IOs? 2) What factors influence branding? and 3) How does 
branding relate to the performance of IOs? Researching branding in IOs can 
involve the use of a variety of sources, for example: interviews with various 
stakeholders, such as United Nations (UN) staff and branding companies, 
or the analysis of branding-related documents of UN organizations, such as 
brand manuals (e.g., https://brand.ilo.org). 

To understand what constitutes branding in IOs, interviewees are asked: 
what is branding in your organization or how is your work related to brand- 
ing? These questions elicit information on branding units, such as a specific 
entity or a campaign, as well as the related branding elements (e.g., logos or 
names) used as part of these units (for an overview of branding elements, see 
Wheeler 2018). For example, the secretariat to the UN Framework Conven- 
tion on Climate Change (unit) was given the name “UN Climate Change” 
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Examples of branding manifestations from the United Nations system 
Branded units (organizations and campaigns) and their branding elements (names and logos) 


Branding elements of organizations Branding elements of campaigns 


@) United Nations @ 


SUSTAINABLE 
DEVELOPMENT Logo for the 17 goals of 


United Nations UN Framework Convention on 
©) ClimateChange Climate Change (secretariat) G@eALS ARN ane 
A Icons/logos of 
A Fa SS individual Sustainable 
u n l cef Es) UNICEF = Development Goals 


Figure 5. Examples of branding manifestations from the United Nations system 


with a corresponding logo (both branding elements). These units and ele- 
ments are what I call branding manifestations (see fig. 5 for examples). 

‘The second question explores the factors that shape branding. Questions 
on why an organization chooses its name or its logo help explain how an IO’s 
characteristics and context influence branding manifestations. One such fac- 
tor may be funding. Some agencies in the UN system, such as UNICEF, 
are considered to receive more funding from private donors due to their 
perceived status as entities not closely associated with the UN.' In keeping 
with this argument, maintaining its current name would be more beneficial 
for UNICEF, rather than entertaining a name change to “UN Children.” 

Finally, the third question identifies the relationship between brand- 
ing and IO performance. For example, effective branding may lead to a 
better understanding by stakeholders of the UN’s work (Tschauko 2017), 
greater trust, and recognition of an organization’s standing, all of which may 
enhance performance. 
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BOX O 


Artifact Analysis 


Julian Eckl 


International organizations could not exist without the artifacts through 
which they manifest and that range from buildings via official records and 
badges to accounting software. Artifacts are ubiquitous but, with the exemp- 
tion of artifacts that feature an inscribed text (i.e., documents), they tend to 
be overlooked. Artifact analysis rectifies this neglect. The analysis (Lueger 
and Froschauer 2018) is guided by questions such as the following: How 
can the artifact be described? What can be done with it? What effects does 
it have on people? 

An artifact that struck me in my research on global health governance 
are “traffic lights” (see fig. 6). They are omnipresent at the assembly of 
the World Health Organization (WHO) but can also be found at other 
decision-making bodies. Attending the assembly, I was intrigued by the fact 
that the traffic lights were in continuous use in this setting but rarely spoken 
about. When I started to document the functioning of the traffic lights and 
to time the individual phases, I was met with disbelief. The participants took 
the traffic lights for granted while analyzing such an artifact helps to unearth 
some of the implicit assumptions concerning the social relations that they 
enable and reinforce. 

In particular, the traffic lights can be interpreted as constituting a calm 
and indiscriminate way of governing the governors and as a material mani- 
festation of the considerable time pressure under which governance bodies 
work. To begin with, traffic lights sequence speakers’ interventions into three 
phases: the first and longest phase could be called “you can talk” (green); the 
second and transitory phase could be called “you have to wrap up” (yellow); 
the third and unwanted phase could be called “you should have stopped” 
(red). Moreover, traffic lights display these phases to everyone in the room, 
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Figure 6. Traffic light 
as used in meetings 
at the World Health 
Organization 
Source: Julian Eckl 


which puts speakers under pressure, assures the audience that the time limit is 
taken seriously, sends the message that country delegates are treated equally, 
and provides the meeting’s chair with shared visual evidence on whether 
speakers may be plausibly requested to conclude (or forced to stop). Unlike 
a sound signal, traffic lights can be used to indicate the (approaching) end of 
the speaking time without immediately disturbing the speaker. Finally, traf- 
fic lights are a constant reminder that time is of the essence and that timing 
is privileged over other ends such as elaborate deliberation. 

A closer engagement with artifacts, however, does not only provide an 
important perspective on present practices. It also alerts to their historical 
emergence. In the case of the traffic lights, their omnipresence is a tangible 
consequence of decades-long reform efforts aimed at making WHO gov- 
ernance “more efficient” (Eckl 2017). Regarding general discussions in the 
plenary, no time limit had been set initially but speaking time was reduced 
incrementally to five minutes that were long common in the years before the 
COVID-19 pandemic; this amounts to a speech that is only as long as the 
text in this box since delegates are meant to read slowly to allow for simul- 
taneous translation. When the first in-person assembly since the outbreak 
of COVID-19 took place in 2022, time pressure was increased even further 
and plenary statements were restricted to three minutes as had been com- 
mon for statements in committee meetings for quite some time. 
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CHAPTER 10 


Document Analysis 
A Praxiographic Approach 


Christian Bueger 


International organizations (IOs) are preoccupied with the writing of docu- 
ments and the handling of paper. Whether these are internal notes, non- 
papers preparing meetings, meeting records, or more official public-facing 
documents, such as communiques, statements, and resolutions, documents 
are the main material and artifacts of IOs. Analyzing documents as the key 
artifact of global governance has the potential to provide important insights 
into how organizations and processes work, why and how they have effect, 
and how power and authority are distributed. Praxiographic analyses study 
the practices around documents, that is, their practical production and writ- 
ing, but also how documents become embedded in wider practices and use. 


What?! 


The increasing scholarly attention for the inner workings and practices of 
international organizations has left a surprising gap. Few seem to focus on 
what is arguably the main activity of IOs: the production and circulation of 
documents. IO documents are conventionally analyzed with either methods 
of text or discourse analysis. 


1. For comments and suggestions on this chapter I am grateful to the editors, the partici- 
pants at the Geneva Workshop that led to this volume, and participants at events at Cardiff 
University and the National University of Singapore where the core ideas were presented. I 
thank Humphrey Asamoah for editorial assistance. 
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Text analyses tend to treat documents as “mirrors” of either actions or 
beliefs. The focus is on the factual content of the document and analyses 
concentrate on what is “in it.” We read the opinions and claims in docu- 
ments as being representative of what the organization thinks and does. 
Meeting minutes are taken as documentation of how the meeting actu- 
ally occurred. Policy documents or declarations are understood as what is 
being implemented on the ground. The problem with such an understand- 
ing is that the document might not reflect actual practices—what is being 
done—or it might be a misleading or inaccurate record. 

In discourse analyses texts are interpreted as articulations and repre- 
sentations of larger structures of meaning. The focus turns to the language 
and expressions used in the document in order to understand assumptions, 
worldviews, and interpretive frames that structure the thoughts and actions 
of the document’s authors. Various forms of hermeneutics, interpretative 
content analysis, or discourse analysis (see chapter 11—Discourse Analysis) 
draw on such an understanding. We then analyze concepts, metaphors, nar- 
ratives, relations between words and sentences, or particular assumptions in 
order to interpret them in terms of culture, discourse, or other larger con- 
tainers of meaning that structure the organization. 

Both of these modes of analysis produce important insights for under- 
standing IOs. Yet they need to be complemented for a number of reasons: in 
stripping the document from the practices surrounding it, they miss out on 
insights on how documents actually are produced and used. They tell us too 
little about what was required to produce the text and the politics and power 
relations that were part of that process. Moreover, they remain too vague or 
abstract when it comes to claims of why and how the documents matter— 
what effects they have and what difference they make. 

The methodological perspective that restores these dimensions is that 
of praxiography. Praxiography is the methodology that draws on insights 
from practice theorizing (Bueger 2014). Practice theorizing posits that 
practices—organized patterns of doing and sayings—should be the main 
unit of analysis (Bueger and Gadinger 2018; see chapter 27—Practice Analy- 
sis). Praxiography hence aims at understanding practices and the elements 
that compose them (such as implicit practical knowledge, explicit norms 
and rules, sayings, bodily movements, artifacts). Documents are understood 
as one element within such practices. Our focus turns to the practical activi- 
ties of making documents, and how they become an element in other prac- 
tices, are interpreted and read. From such a perspective, we can analyze how 
documents are produced and when and why they become powerful devices 
in global governing. 
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Documenting is so central to the majority of global governance processes 
that it can be understood as an “anchoring practice.” According to Neu- 
mann and Sending (2010), anchoring practices structure other activities 
and practices. Meetings for instance are prepared and structured through 
documents and allow actors to “go on record” and ensure that they agree on 
statements and interpretations. 

Studying the production process reveals insights into the inner working 
of a formal or informal IO and what kind of relations it is part of. This leads 
to a better understanding of organizational identity, culture, and norms, as 
well as the practical knowledge that informs organizational practice (Riles 
2006; Freeman and Maybin 2011; Cooren 2004). In so far as document- 
ing is an essential practice of many contemporary organizations, we learn 
how organizations come into being and sustain their existence (Neumann 
2007). Studying the authorship of documents and negotiation processes, 
moreover, allows researchers to understand the underlying distributions of 
power, authority, and expertise within an organization and its environment 
(Cooren 2004). 

‘The praxiographic study of documents also provides important answers 
to a second question, namely how do documents have an effect? Discourse 
analytical approaches posit that the answer lies in the claim that texts are 
articulations of broader structures of meanings; it is these structures that 
produce the effect. Praxiography develops a different argument. Documents 
become powerful devices through being used in practice. They exert an 
effect in being circulated, read, cited, and referred to, publicly interpreted 
or used in other ways. To study the effects of texts is hence to investigate the 
practices in which texts are used. Through such analyses we gain an under- 
standing of the authority and position of the organization that authored 
the document in larger textures of practices. Analyses will be able to dem- 
onstrate how organizations achieve structural and orchestration effects, but 
also how the document becomes used as a tool of coordination due to it 
being circulated as a material object. 


How? 


Praxiographic document analysis implies a detailed reconstruction of prac- 
tices of producing and using text (see chapter 25—Process Tracing). In the 
majority of cases, this implies identifying a particular kind of document 
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that occupies an important role in an organization. For instance, in the case 
of the UN Security Council, such documents might be its resolutions. Yet 
there are also more mundane or secretive documents, such as sanction lists 
(Sullivan 2020). Other studies have focused on, among others, ministe- 
rial speeches (Neumann 2007), UN Conference declarations (Riles 2006), 
bureaucratic forms (Walters 2002; Hull 2012), or policy documents (Free- 
man and Maybin 2011). 

In my own research on the global governance of piracy, I have studied 
how international actors succeeded in curbing the problem of Somali piracy 
within a short time span of five years. Addressing this question through 
a multiyear praxiographic project led me to two kinds of documents that 
acted as powerful coordination devices: 1) a best practice document that 
was key to regulate the practical relations between the maritime transport 
industry and military actors (Bueger 2018), and 2) the communiques of the 
Contact Group on Piracy off the Coast of Somalia (CGPCS), which was 
the main informal global governance body dealing with piracy. These docu- 
ments were crucial for assembling and coordinating the broad array of more 
than eighty international actors under a common strategy. 

My study of the two kinds of documents was based on a long-term mul- 
tiyear endeavor that implied deep immersion in the CGPCS. I participated 
in all the CGPCS’s various meeting formats for more than five years. This 
gave me the opportunity to not only observe the meetings and negotiations 
but also participate in the drafting of the communique. Eventually, I also 
became the pen holder for one communique in 2017 when I worked as one 
of the assistants of the CGPCS chairman. This immersion provided me with 
both access to people and immediate experiences. While such deep immer- 
sion may be ideal, a study of the making of documents, however, essentially 
boils down to asking the right questions. The most important is perhaps the 
task of dissecting the controversies that had to be resolved in order for the 
document to be concluded. 

Document making is often a much more controversial process than what 
can be seen by looking at the text. The drafting process in essence is a pro- 
cess of resolving controversies around issues, such as: what characterizes the 
problem that needs to be addressed? What is part of the problem and what 
not? Who should be mentioned in the document? Which actors should be 
given a particular role? Whose activities should be highlighted and who's 
not? Which actions should be welcomed, criticized, or condemned? What 
should be concluded in terms of what different actors ought to do? My 
reconstruction of the making and revisions of the best practice document 
revealed key divergences over who has the authority to decide when and 
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where a high risk for piracy exists and the special measures outlined in the 
document should be followed. This entailed different claims to expertise 
and authority between the transport industry, insurance industry, naval mis- 
sions, and regional states that had to be aligned. The negotiation of the 
CGPCS communiques entailed debates such as on the urgency of the piracy 
problem, how much of the solution was to be found within the state of 
Somalia, in a regional response, or in activities at sea. Participants compete 
over being mentioned in positive terms and being recognized as having pro- 
vided significant input in the drafting process. Diplomatic skills, language 
skills, knowledge of the history and law of counterpiracy, regional political 
knowledge, and technical knowledge of what is feasible all become impor- 
tant resources in that process. 

The success and working of counterpiracy, and especially the role of 
the informal global governance forum, the CGPCS, cannot be understood 
properly without reconstructing the role of these documents. Indeed, how 
much the communique structures the work of the CGPCS was, to my sur- 
prise, obvious when I attended my very first plenary meeting. Much of what 
happened at that meeting and all the others was in one way or another 
already scripted by a draft of the document written before the meeting even 
started. Much of the time at the meeting was given to completing this draft. 
In following the process, I learned about the complicated techniques (“skel- 
eton drafts,” “bilaterals,” or “reading procedures”) that the group employed 
to align actors and to agree on the document. With some variation, many 
processes of multilateral diplomacy within international organizations or in 
the various informal governance formats, such as groups and conferences, 
operate in a similar manner. The document is the key to understand how 
these operate. 


What Challenges? 


Treating the document as a governing device and as an object of analysis in 
its own right provides important new lenses on IOs. This implies researchers 
should take the document as an element of practice and study practices of 
writing, negotiating, circulating, reading, or interpreting. Like other meth- 
ods, the praxiographic study of documents faces some challenges (further 
discussed in Bueger 2021). 

Studying documents in such a way requires a well-negotiated proximity 
to the practices (and those involved in the production of the documents). 
To appropriately understand such practices, one has to leave the desk (physi- 
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cally or virtually) and go to the sites where the document is made and used. 
This implies that researchers should first follow the document and identify 
those sites. This might be intricate as writing can be distributed. In the case 
of the CGPCS, the most important site was the plenary meetings. Yet early 
drafts of the communique had been discussed already elsewhere. Thus the 
potential number of sites is infinite; however, some will matter more than 
others. Yet this is an advantage. Getting invited to these sites might imply a 
lengthy negotiation process. If there is a difficulty, one can start with a more 
marginal site and then increasingly get closer to those more central to the 
process. 

‘The same principle can also be applied to interviewing those who are 
partaking in the production, circulation, and use of a document. Identifying 
these people, getting their attention, and getting a slot on their busy sched- 
ules takes effort (see chapter 5—Semistructured Interviews). These efforts are 
not that different from any other interviewing process; what can be chal- 
lenging is to select interviewees according to their role in document produc- 
tion, circulation, or use. In addition, any interviewee will only have partial 
knowledge of the practices of documents. Hence one needs to piece the 
puzzle together. The worlds of international relations are a constant stream 
of documents; the practices that matter are globally and spatially distrib- 
uted. One therefore has to carefully select which documents to focus on. 
Asking which documents are powerful devices and make stuff happen is an 
important starting point. 


To Go Further 


Bueger, Christian. 2021. “Conducting ‘field research’ when there is no ‘field.’ Some 
notes on the praxiographic challenge.” In The Political Anthropology of Internation- 
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Freeman, Richard, and Jo Maybin. 2011. “Documents, practices and policy.” Evidence 
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BOX P 


Semiology of Websites 


Camille Rondot 


Analyzing international organizations (IOs) and their digital productions 
face a number of challenges. This box presents a series of methodologi- 
cal tools inspired by semiology and discourse analysis applied to digital 
productions. 

Semiotic approach—methods inspired by the semiological work of 
Roland Barthes (1957)—allows researchers to analyze symbolic dimension 
of IOs and develop a deeper understanding of their digital communica- 
tion. This is about questioning denotation (literal meaning) and connota- 
tion (associated meaning) of each sign of digital production. With such an 
approach scholars may grasp four dimensions in IOs’ digital media repre- 
sentation: first, by questioning the role of technical devices in IO commu- 
nication (computer, smartphone, etc.); second, by identifying the different 
components displayed on the screen (“screen writings”) (Souchier and Jean- 
neret 1999); third, by being aware of the constraining role of software or 
applications (with PowerPoint you cannot do what you want, with Twitter 
you have to be concise, etc.); and fourth, by developing a reflective and criti- 
cal approach. Hence “digital” does not mean “revolution”; rather it means 
“transformation” (unlike Pressbooks, the content does not disappear when 
the computer crashes) and continuation, with the visual and literate culture. 
A methodological approach thus makes it possible, through the exploration 
of icons, symbols, and clues, to analyze the meaning and the effects of mean- 
ing of what is displayed on the screen. It invites researchers to explore a set 
of questions and to engage in an interpretational effort. 

‘The website, as an informational and communicational unit defined by 
a URL (for instance, https://www.unesco.org/fr), becomes a particularly rel- 


158 


Semiology of Websites 159 


evant place to understand how the organization conveys meaning and shows 
its missions. Concretely, I suggest compiling a detailed inventory of every 
digital device and their structure. For instance, in my research on UNESCO’s 
website (Rondot 2015), I proceeded in three different stages. First, I created 
an arborescence (tree structure of the website) with the names of pages and 
URLs. This allowed me to model a representation of the organization with 
hierarchical links among a set of contents. Second, I took screenshots of each 
page and wrote the title, URL, and date. Third, I collected hyperlinks. This 
allowed me to highlight the importance of the hypertextual dimension on 
the website: between legitimation effects (a link to a state, another IO can 
show a connection between actors) and Search Engine Optimization logic 
(many internal links are important for SEO). 

Last but not least, to fully understand the specificities of IO’s digital 
communication, it is necessary to take into account nondigital devices. It 
has to be considered as part of a wider communication strategy. It could also 
be useful to set out the historical background of the website by using the 
“wayback” machine (https://archive.org/web/). It helps highlighting com- 
munication transformations and the role of design. 
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BOX Q 


Analyzing Tweets 


Matthias Hofferberth 


International organizations (IOs) have “gone digital” in the last ten years 
(Ecker-Ehrhardt 2020). As it stands, whether on Facebook, Twitter, Ins- 
tagram, YouTube, or other outlets, all IOs today have some form of social 
media presence (Twiplomacy 2018). Through this latest form of engagement, 
IOs have a new tool at their disposal to directly relate to their stakeholders 
to signal responsiveness, influence public deliberation, and overall increase 
their perceived legitimacy. Research on social media in International Rela- 
tions (IR) in general, and on IOs and their digital diplomacy in particular, 
however, is still at an early stage (Bjola and Zaiotti 2020). Using Twitter and 
analyzing tweets can be an important way forward to explore and account 
for digital diplomacy, simply because this medium is politically salient due 
to its users and topical foci. Furthermore, some of its characteristics prove to 
be advantageous for research. With its brevity, conciseness, and immediacy, 
Twitter can be considered “amplified” social media and therefore an impor- 
tant tool for studying organizational identity and collective agency (Murthy 
2012). Equally important, from a methodological perspective, is the easy 
access that Twitter offers through its application programming interface 
(API). This allows researchers to download, archive, and code tweets in real 
time. Various tools to do this are integrated into statistical computing soft- 
ware (e.g., R) or available online (e.g., Twitter Archiver). In addition, there 
are commercial providers, such as Vicinitas Analytics, which provide com- 
plete data sets. These tools, with a little bit of skill and knowledge, can be 
tailored toward the individual project to provide meaningful data. In my 
own research, I have collected UN tweets from various Twitter accounts and 
applied a qualitative content analysis focused on content, purpose, audi- 
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ence, and organizational self-assessment (Hofferberth 2020). A major chal- 
lenge was to determine which accounts constitute the UN Twittersphere. 
Distinguishing institutional (@UN) from individual (@antonioguterres) as 
well as tweeting on behalf of the organization (i.e., official handles) from 
those toward the organization (i.e., state missions and NGOs) proved to be 
helpful. This allowed the project to organize different accounts and compare 
their respective focus and impact. Reconstructing IO tweets in their respec- 
tive Twittersphere can thus be utilized to determine organizational self- 
understandings, emphasis put on different issues, and overall engagement 
and projected authority as a global governor. In other words, tweets provide 
an important source to understand IOs and the narrative with which they 
“go public.” If anything, as IOs continue to expand their social media pres- 
ence and produce more material to analyze, scholars interested in IOs have 
new digital ground to cover. 
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CHAPTER 11 


Discourse Analysis 


Audrey Alejandro, Marion Laurence, and Lucile Maertens 


International organizations (IOs) are the setting for the production and dis- 
semination of overlapping discourses. They annually publish thousands of 
documents in which one word can be debated over years of negotiations. 
Discourse analysis assists scholars to grasp both internal processes of dis- 
course production within organizations and the impact of IOs’ discourses in 
the making of global politics. 


What? 


Since the “linguistic turn” in the 1960s, the social sciences and humanities 
have increasingly made discourse an object of social enquiry, challenging the 
idea of language as a neutral medium of communication. The concept of 
discourse—broadly defined as language in context—highlights processual 
and interactional dimensions of meaning-making, foregrounding the copro- 
duction of discourses and sociopolitical configurations. Scholars have devel- 
oped a rich theoretical and methodological tradition, often referred to under 
the umbrella term “discourse analysis” (DA). More than other textual meth- 
ods focusing on manifest dimensions of language (like content analysis, see 
chapter 17—Computerized Text Analysis), DA examines the implicit dimen- 
sions of language and how they interact with invisible aspects of world poli- 
tics. DA focuses on contextual and productive effects of words and speech: 
scholars can study processes that are otherwise hard to assess empirically like 
norm creation. It enables researchers to study naturally occurring discourses, 
such as in legal documents and policy reports, as well as discourses generated 
through elicitation techniques such as interviews and surveys. 
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A variety of methodological options are available for IO researchers 
interested in using DA. Rather than a strict rule book, DA encompasses 
both a theoretical-ontological approach—the idea that discourse plays a role 
in society and world politics—and a flexible set of methods to empirically 
investigate this idea. DA allows researchers to approach discourses either as a 
source, to identify what they reflect (e.g., norms, ideologies), or as a practice, 
to investigate what they produce (e.g., discriminations, identities). 

DA proposes two main approaches. The first, following Michel Foucault, 
takes a discourse (or several) as the main unit of analysis, such as the emer- 
gence of a new discourse defining madness as an illness legitimizing the insti- 
tutionalization of psychiatry (1965). The second approach, well-established 
since the 1980s, focuses on the linguistic dimensions of discourse to inves- 
tigate how different wordings can reflect and produce different sociopoliti- 
cal orders. For instance, Critical Discourse Analysis investigates how the 
use of linguistic devices, like the passive voice, produce power relations and 
prejudice. 


Why? 


In IO studies and IR more broadly, many scholars engage with discourse 
without explicitly claiming to do DA. This highlights the potentially broad 
appeal of DA, but also trepidation about using a method that seems to 
require mastery of complex ontological and epistemological debates. We 
identify four overlapping types of puzzles that DA can help uncover. 

First, DA provides tools to analyze agenda-setting and framing processes 
by unpacking the ways IOs represent global problems. IOs are discursive 
sites where meanings are negotiated in interaction, and interfaces through 
which representations circulate across the world. For instance, exploring the 
“social life of text” in climate negotiations, Aykut analyzes the mechanisms 
through which outcome documents exclude different issues and framings, 
which then translate into governance features (2017). Moretti and Pestre 
stress the role of derived abstract nouns in World Bank reports, which turn 
actions and processes into “abstract objects” (2015: 90). 

Second, DA is useful to investigate global norm setting and the establish- 
ment of governance regimes. Epstein, for instance, uses DA to show the 
normalization of a global antiwhaling discourse: she denaturalizes what is 
assumed to be the “right” discourse (i.e., antiwhaling) by tracing its his- 
torical development and the symbols embedded within it (2008). Likewise, 
Zanotti (2008) conducts a genealogy of UN texts on peacekeeping in Haiti 
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and shows how dimensions of UN discourses inherited from the Enlighten- 
ment led to unintended consequences in regard to peacebuilding practices 
and democratization aspirations. Krook and True use a discursive under- 
standing of norm change to explain outcomes in gender-balanced decision- 
making and gender mainstreaming in the UN campaign to promote gender 
equality (2010: 105-6). 

Third, DA helps assess how actors use [Os as discursive arenas and the 
ways [Os engage in self-legitimation. Donahue and Prosser analyze discourses 
occurring in the UN as “diplomatic speech-making” (1997: 1). Shepherd 
maps the construction of “civil society” in UN peacebuilding discourse, 
showing how that discourse (re)produces the UN itself as the “legitimate 
knower of peacebuilding practice” while casting local communities as 
“known objects” (2015: 887). Likewise, von Billerbeck shows that peace- 
keeping officials in the UN Secretariat maintain a cohesive and legitimate 
organizational identity by relying on discourses that “simplify” and “excep- 
tionalise” their work (2020: 1). 

Finally, DA critically examines prevailing power dynamics, showing how 
discourse can “perpetuate, institutionalize, and legitimate asymmetries of 
power” (Holzscheiter 2014: 150). IOs normalize and legitimize compet- 
ing discourses, which represent and benefit different types of actors. For 
example, Rist challenges the supposedly neutral nature of IO “reports” by 
showing that their prescriptive character is reflected in the use of the impera- 
tive mode, the verb “must,” and passive forms (2002: 39). Pratt uses DA 
to critically examine UN Security Council Resolution 1325. She argues 
that the resolution reproduces structures of power that embody “gendered, 
racialized, and sexualized hierarchies,” which in turn underpin hegemonic 
discourses and security practices of the post 9/11 era (2013: 780). 


How? 


Many researchers will select a specific DA method by emulating the work 
of scholars who explore similar research questions or by matching their 
approach to an existing theoretical framework (e.g., poststructuralist DA 
for poststructuralism, or genealogy for power-knowledge). For instance, 
researchers may draw inspiration from Critical Discourse Analysis (CDA), 
which provides an established approach for analyzing discourse through 
three elements (Fairclough 2003): the genre or the form that structures dis- 
course, like a news report, political speech, resolution, and so forth; the style, 
which considers the “manner of doing things” linguistically (Balker 2011: 
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141); and the ideational content of the discourse. Traditionally, this is done 
using a qualitative approach, although recently some scholars have sought 
to combine CDA with quantitative methods. Felli (2016), for example, ana- 
lyzes the World Bank’s discourse on resilience by documenting an increase 
in occurrences of the lemma “resilien-” in the annual World Development 
reports, which he then codes according to Fairclough’s three elements. 

However, in many cases, there is no ready-made DA method matching 
the specific needs of one’s research. This is common when a scholar’s research 
question flows inductively from specific texts (e.g., IMF working papers) 
or from a puzzling discursive event such as a UN General Assembly debate 
that unfolds in a surprising way. In such cases, we encourage researchers to 
create a bespoke method for themselves by making innovative use of the DA 
tools already available in a four-step process. We refer to this sort of “do-it- 
yourself DA” as “Bespoke Discourse Analysis.” 


Step l: Corpus Construction and Data Collection 


A. Constructing the corpus: a corpus is a large and structured set 
of documents constructed for a specific purpose. Defining it is 
paramount and requires trade-offs between breadth (size of the 
corpus) and depth (number of discursive mechanisms examined 
and richness of contextualization). These sampling trade-offs may 
depend on issues around access and availability. Some texts, like 
UN Security Council resolutions, are easily accessible online and 
translated into six languages. Others, like disciplinary proceedings 
or the results of an internal UN investigation, are much harder 
to access. Researchers need to carefully consider the institutional 
logics and power dynamics in which discourses are embedded; 
the important thing is to be transparent about challenges and 
justify one’s choices. 


B. Selecting the data sources: the type of data being collected should 
match the research question, which can be revised and adapted 
during the project. Conclusions must then only be inferred 
based on the available material. Multi-method designs are often 
required to address discursive questions that textual analysis alone 
cannot answer. For example, researchers cannot assess the impact 
of policy discourse from the African Union nor claim to capture 
authors’ intentions based only on the textual analysis of policy 
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documents. Researchers may collect such additional data them- 
selves or draw from the existing literature. 


Step 2: Mapping the Context and Preliminary Analysis 


A. Mapping the context: DA can be understood as the interpreta- 
tion of texts and speech in the context in which they are produced 
and received. It is therefore important to understand and mobi- 
lize elements of the context to produce a rigorous interpretation. 
Different dimensions of the context need to take into account (i) 
the situation of utterance (who speaks to whom? When? Where? 
About what?); (ii) the sociohistorical context (institutional, socio- 
political, positional, relational context of the documents or the 
participants, etc.); (iii) the textual context (genre, paratext, inter- 
text); and (iv) if relevant, the sensorial context (images, moving 
images, and sound). 


B. Doing preliminary analysis: researchers should ensure that they 
understand what the text (or document) explicitly aims to com- 
municate before starting to explore its implicit dimensions. For 
example, what event, experience, or policy is the speaker referring 
to? What is the main argument the speaker is trying to convey, 
and so on? 


Step 3: Identifying Relevant DA Tools 


Analytical tools for DA include, for example, lexical fields, metaphors, or 
subject positionings. There are two main ways to identify which DA tools 
are relevant for one’s project: 


A. Doing a literature review jointly searching for both topics related 
to the project and DA (e.g., “racism” AND “discourse analysis”) 
to see how other scholars have tackled related projects and which 
tools they have used to conduct their analysis. 


B. Combining “immersion” phases of reading and rereading the cor- 
pus (Gill 1997: 144), with consultation of “toolkit-style” hand- 
books that present available tools (Gee 2011; Balker and Ellece 
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2011), to define the linguistic processes that one may intuitively 
grasp through reading. 


Step 4: Iterative Interpretation and Writing Up Results 


A. Interpreting: researchers use selected DA tools and the contextual 


elements to delve into the iterative and systematic interpretation 
of the corpus. For instance, while the use of the passive voice 
obscures agency, the effects of passivization differ depending on 
the context: vulnerable populations can be denied agency versus 
responsibility for harm can be diffused to promote reconciliation. 


B. Writing up results: this requires simultaneous mobilization of 
excerpts from the text, the tools used to analyze them, and the 
context to support each argument. 


Table 5. Discourse Analysis Checklist 


10. 


Do you have a research question compatible with the use of discourse analysis? 

Do you justify why discourse analysis is the most adapted method of analysis for your 
research project? 

Have you clearly identified and defined the method(s) of discourse analysis and 
discourse analysis tools you are using? 

Do you justify how the chosen method of discourse analysis and discourse analysis 
tools are aligned with your research question and conceptual framework? 

Is the chosen method of discourse analysis aligned with the type of material and data 
you aim to analyze? 

Do you justify criteria of inclusion and exclusion and sampling strategies of your 
corpus? 

Is the corpus aligned with your case, literature review, research question, and method? 
Do you provide elements of contextualization about your corpus (e.g., the context of 
production and reception of documents) so that the reader can understand the value 
and role of the chosen sources in their context? 

Is each argument you put forward in your analysis both supported by elements of 
discourse you identify in your data and elements you draw from the context and 
literature? 

Do you explicitly reflect on how your position/trajectory/socialization might have 
influenced your analysis and present the actions you took to address these reflexive 
insights? 


Note: Table 5 has been adapted from this webpage: https://www.audreyalejandro.com/blog---the 
-methodological-artist/checklist-questionnaire-when-revising-a-research-assignmentproject 
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DA provides a flexible set of tools to investigate the role that discourse 
plays in society. Adjustments may be required but following these four steps 
should provide researchers with a roadmap for adapting DA to their specific 
needs. The Discourse Analysis Checklist provides a series of generic ques- 
tions to guide the use of DA in a research project. 


What Challenges? 


Some of the challenges that come with applying DA to the study of IOs 
result from the method itself. Others are related to the complexities of dis- 
course production within IOs. 

First, DA requires strong theoretical, epistemological, and ontological 
coherence. Many theories of discourse challenge concepts of agency and 
intentionality, meaning that scholars must ensure that their theoretical 
framework aligns with their ontological and empirical claims. Implicit ele- 
ments and effects of discourse may be lost on those who produce and receive 
them. For instance, World Bank officials may not intend to depoliticize pov- 
erty when setting up quantitative indexes and standards that reduce it to a 
technical issue (Louis and Maertens 2021). Claims about agency and inten- 
tionality should demonstrate awareness of these dynamics. Second, research- 
ers are social agents who produce—and are socialized into—particular dis- 
courses: reflexivity helps account for the sociological characteristics and 
trajectory that lead to interpretation (see box y—Reflexivity in Practice). This 
means considering the extent to which one’s own knowledge, experiences, 
and assumptions are shared by the social agents that one studies. Implicit 
references and connotations are not universal (e.g., what “security” means), 
and specific discursive elements do not make sense in all social contexts (e.g., 
the meaning and value of metaphors are localized). 

IOs are complex settings for discourse production and circulation: clear 
theoretical and methodological justifications are useful when defining the 
scope of “IO discourses.” For example, the rationale for studying official 
MERCOSUR reports and web pages can be minimal, but the extent to 
which discourses produced by MERCOSUR employees represent “MER- 
COSUR discourse” needs to be demonstrated on a case-by-case basis. Simi- 
larly, identifying the authors of an “IO discourse” is challenging because of 
corporate and collective authorship and runs the risk of reifying the organi- 
zation and concealing internal dynamics. Common characteristics in IO dis- 
course also need to be problematized—like the taken-for-granted labelling 
of “local” versus “international” discourses. One helpful strategy is to learn 
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as much as possible about how different texts and documents are produced 
within a given IO and adjust empirical and theoretical claims accordingly. 
All case studies require contextualization to guard against one-size-fits-all 
interpretations of IO discourse as institutionalized discourses might have 
different meanings and effects. 
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BOX R 


Studying Ideas 


Olivier Nay 


Most international organizations (IOs) are engaged in normative activities. 
They produce, promote, and share core ideas that frame multilateral agen- 
das and legitimize international interventions. These ideas are mental con- 
structs drawn on scientific knowledge, technical expertise, and ideological 
preferences. Most prominent ones are condensed into generic concepts— 
such as “sustainable development,” “human security,” or “fragile states’— 
that are often used by experts and decision-makers as mental shortcuts to 
think about social problems. They become international norms when they 
are officially converted into prescriptive standards for policy development. 


The Social Life of Ideas 


The study of ideas in IOs refers to the analysis of the characteristics and attri- 
butes of core policy categories that get traction and circulate in the system 
of multilateral organizations. Such a definition implies that ideas are not 
disconnected from practical activities and political intentions. Rather they 
emerge, spread, and gain importance in public discourse as “social produc- 
tions” that are modeled by the different actors operating in and around IOs 
(see chapter 11—Discourse Analysis). 

‘The study of ideas can explore two intertwined fields of observation: first, 
IO secretariats, which are subject to permanent “knowledge battles” involving 
their bureaucratic units, member states, and expert groups; second, the IO 
ecosystem in which many actors operate as knowledge entrepreneurs (policy 
communities, expert networks, universities, and think tanks, among others). 
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Ideas are far from remaining stable in time and space. Once coined, they 
travel as they are subjected to multiple appropriations that redefine their 
scope and meaning. Ideas have thus a social life, just like objects (Appadurai 
1986). They may even have several lives, as they are exposed to various stra- 
tegic uses and cultural acclimatization depending on which actors appropri- 
ate them. 


Following the Idea 


The approach “follow the idea” is a heuristic method to explore the life of 
ideas among international institutions. Inspired by the “follow-the-thing” 
perspective initiated by Appadurai’s ethnography (Harvey 1990), it consists 
of following the circulation of a particular idea and its multiple uses. In the 
context of IOs and their linked ecologies, it entails investigating the chrono- 
logical, spatial, and social trajectories of the idea: emergence and circulation 
in professional or policy networks tied to international bureaucracies; adop- 
tion by IO staff, units, and executive bodies; dissemination throughout the 
IO ecosystem; normative adaptations, appropriations, and diversions from 
stakeholders, partners, and competing entities; decline and demise of the 
idea; and potentially the transition toward a new prevailing idea (Nay 2020). 

The methodologies used to track the development of ideas may concen- 
trate on textual materials available in organizations’ archives (e.g., public 
records, statements, reports, resolutions, technical documents), as well as 
communication materials (e.g., memos, letters, blogs, web content, audio, 
and visual records). Quantitative methods (e.g., content analysis, computa- 
tional methods) offer multiple grounds to study circulating ideas, including 
their prevalence in various lexical fields attached to organizations, and their 
discursive change over time. 

Nevertheless, only qualitative methods (e.g., interviews, questionnaires, 
observations) allow for ethnographic explorations of the social contexts within 
which ideas are produced and fostered. They offer major insights into actors’ 
strategic use, power games, and knowledge battles within international sec- 
retariats, as well as the influence of knowledge networks in their direct envi- 
ronment. Situated approaches reveal that ideas are social constructs that are 
constantly being locally reinvented in the daily life of each IO. 
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CHAPTER 12 


Statistics and Quantification 


Roser Cussó and Laure Piguet 


Quantified information is one among the many daily productions of IOs. 
From trade statistics to unemployment rates, these institutions make avail- 
able to a “world community of users” data on almost every aspect of the 
social and physical world. How can a researcher approach these statistics? 
What are the particularities of studying data produced by IOs as compared 
to the study of other IO productions, or of national statistics? What does 
this research tell us about the functioning of IOs and, more broadly, about 
statistical expertise as an instrument of power and influence? The sociology 
of quantification offers useful methodological guidelines for the researcher 
who aims at opening the “black box” of the statistical production of IOs. 


What? 


The sociology of quantification (Desrosiéres 1998; 2008) was developed 
beginning in the 1970s, largely in response to the work of Pierre Bourdieu. 
For Bourdieu, analysis of statistical categories was essential because “mea- 
surement and tools of measurement [. . .] are simply theories in action, 
which result from processes, both conscious and unconscious, of construct- 
ing facts and the relations among facts” (Bourdieu et al. 2005 [1968]: 59). 
Quantification refers to a process that translates social or natural phenom- 
ena into statistics or other numerical information. For instance, a country’s 
wealth might be expressed in terms of its gross domestic product derived 
from national accounts. Quantified data thus both constructs and brings to 
light the “poor countries.” Quantification also involves scientific perceptions 
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and policy (e.g., correlating low income and poverty, and thus eventually 
promoting economic growth). When defining quantification, it is pertinent 
to distinguish statistics (figures) from statistical science; the former are ana- 
lyzed by the latter (descriptive, inferential statistics, etc.), as well as by algo- 
rithms and other mathematical models, which have been proliferating more 
recently in computer science. 

‘The quantification-based approach merges political, social, and technical 
processes and practices. It thus calls for interdisciplinary analysis (statistics, 
social sciences, historiography, and knowledge of what is measured, i.e., dis- 
eases, employment, etc.) as well as for specific theoretical bases adapted to it. 
For Desrosiéres, quantification involves both the working out of “conven- 
tions” on the statistical categories, methods, and procedures for data col- 
lection (socially agreed commensurability) and the proper and more vis- 
ible “measurement” (figures) of the quantified “fact.” Intrinsically linked 
to the rationalization of power and knowledge (Foucault 1991) and to the 
strengthening of modern states (Desrosiéres 1998 [1993]), quantification 
“[.. .] creates a new way of thinking, of representing, of expressing the world 
and acting on it” (Desrosiéres 2008: 11). The sociology of quantification 
thus invites us to reflect upon the “co-construction of scientific statistics and 
power regimes” (Speich Chassé 2016: 220). As regards the methods to study 
quantification, they may be of various kinds (archives, observation, profiles/ 
interviews of actors, data, and discourse analysis, etc.) and may need to be 
combined by researchers in order to grasp complex and manifold processes. 

Research on quantification has mostly focused on state statistics. Both 
the study of IO data as a quantification issue and the subsequent proposi- 
tion of adapted approaches are recent (Cussó 2012; 2019; Speich Chassé 
2016; Martin 2023; Piguet 2018; Bemmann 2019). Such approaches are to 
take into account the founding differences between state and IO quantifica- 
tion. States are generally ruled by political alternation and social conflict; 
IOs lean on universal goals and consensus, thus enjoying relative autonomy. 
In this context, while quantification processes may be more democratically 
supervised in states than in IOs, international data is more readily seen as 
politically “disinterested.” This makes IO quantification emerge as a political 
innovation, along with cross-country comparisons and the important role 
taken by technical activities and cooperation. 
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Analysis of international quantification contributes to better understanding 
of the origin, functioning, influence, and evolution of IOs. First, as regards 
their origin, the missions of the League of Nations, for instance, were imme- 
diately accompanied by international statistical programs, and technical 
activity more broadly, which not only contributed to but characterized the 
League’s emerging power. 

Second, quantification studies help to clarify the functioning of IOs, 
since their statistical programs imply the intertwined activity and decision- 
making of different actors in different spheres: intergovernmental (assem- 
blies), transnational (expert commissions, NGOs, unions), and interna- 
tional (secretariats). The “mise en discipline” (in Foucault’s sense) of these 
actors is to be specifically analyzed through technical cooperation activities. 

Third, focused on actors and processes, analysis of quantification helps 
to precise the way IOs wield influence and why the latter is often limited 
and/or has to be nuanced. Indeed, statistics both reinforce IO goals and 
make them depend on intergovernmental cooperation and involvement; for 
instance, global quantified benchmarks on public debt may lead a govern- 
ment to constrain its expenditures, as being under international scrutiny and 
“shaming.” Such benchmarks both construct “global problems” and influ- 
ence national policies. Yet it is only active state participation in the relevant 
IO statistical programs that ultimately allows for, and subsequently legiti- 
mates, IO benchmarks and peer scrutiny. 

Fourth, the study of IO statistical systems contributes to illuminate IOs 
history. For Desrosiéres (2000), national statistical systems are closely related 
to state “historical configurations” such as mercantilism, liberalism, the wel- 
fare state, or neoliberalism. For instance, the early welfare states connected 
comprehensive statistical assessments of poverty and large supply-oriented 
policies on working conditions. A neoliberal state, by contrast, tends to 
link microdata analysis to demand-oriented policies encouraging individual 
incentives and financial efficiency. The evolution of IO statistical systems 
also reveals different IO “historical configurations.” As analyzed by Cussó 
and D’Amico (2005), UNESCO reformed their statistical analysis in order 
to facilitate correlation and country rankings, all more adapted to 1990s 
neoliberal policies. More recently, the development of big data spur flexible 
and “case-by-case” policies (see for instance UN Global Pulse 2018). Indeed, 
such data favor ad hoc technical decisions and increased outputs, as often 
based on multiple and disparate sources. 
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How? 


As expressed by Fabrice Bardet (2018), a quantification-based approach 
should consider three intertwined questions: who produces the quantified 
data? How is it produced? And why is it produced? Their main objective is to 
reconstruct both the characteristics of the quantification being analyzed and 
its underlying power rationale (paradigm and policy). 

As regards the who question, the actors and institutions relating to IO 
data are diverse: the IO personnel, the representatives of member states, the 
experts, the members of the specialized commissions, and any other actor 
who participates in (or influences) the statistical process as, for instance, the 
workers unions within the ILO. Data providers and users also need to be 
considered, especially the national statistical services. Are then to be stud- 
ied actor professions, experiences, and affiliations as well as actor roles and 
power within the statistical production, from data provision to data publica- 
tion and use. 

‘The ow dimension mainly entails analyzing stages and processes of data 
standardization. To ensure cross-country comparability, national statistics 
are to be previously classified following agreed-upon international methods 
and definitions (“conventions”); this is followed by further harmonization 
by IOs (international data analysis and treatment). Indeed, in view of the 
sheer range of national definitions and methods, IOs are first to establish 
common references. As regards labor issues, for instance, in one country 
unemployment may concern people having lost their job, in another people 
actively seeking employment, and in a third people registered by a place- 
ment office. Unsurprisingly the implementation of common methodologies 
is to imply a huge stake for governments and calls for serious cross-country 
cooperation. That is why the IO ow dimension also involves the examina- 
tion of the manifold actor interactions, negotiation, and micro-choices that 
make a qualitative and/or national phenomenon emerge as an internation- 
ally quantified topic. 

Finally, the why dimension refers more specifically to the purposes and 
uses of IO statistics. One of the explicit objectives of the UN Population 
Division, for instance, is to produce “constantly updated demographic 
estimates and projections for all countries, including data essential for the 
monitoring of the progress in achieving the MDGs and now the SDGs, 
developing and disseminating new methodologies [. . .]” (UN DESA). Yet a 
research on quantification is not only to point out that population statistics 
may legitimate UN global discourse on population issues and problems but 
also to unveil IO implicit demographic norms. 
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‘There are different ways to implement a quantification-based approach. 
One possibility is to follow the steps of IO statistical programs, from 


approval of their official objectives to final outputs and uses (Cussó and 


D’Amico 2005). Such a comprehensive analysis calls for a long run research 
project (PhD projects or monographs). Another possibility is to focus on 
selected research questions or issues: on international measurement of labor 


accidents (Piguet 2018), for example, or on a specific labor indicator (Berten 


2019). 


Practically speaking, the following research activities can be identified: 


a) 


b) 


c) 


d) 


e) 


Making an exploratory analysis of the targeted IO statistical 
outputs (online reports with their statistical appendix, technical 
notes, available data, metadata, etc.). They would give an idea of 
the implicit quantification choices made by the IO being studied. 


Identifying the objectives of the statistical activities through anal- 
ysis of the missions of, and decisions made by, intergovernmental 
assemblies and commissions and international bureaucracies, in 
the form of founding charts and statements, but also through 
agendas, correspondence, and interviews (see above). 


Analyzing the formulation of statistical agreements (“conven- 
tions”) that level out the diversity among national definitions 
of the phenomena measured, and thereby ensure comparability 
(observation/interview, minutes of meetings, technical papers, 
guidelines, etc.). 


Reconstructing technical cooperation (participation of member 
states in technical meetings, training missions organized by the 
IOs, nomenclatures, and manuals). Analyzing national statistical 
services can help here to ascertain the extent to which countries 
have reformed (adapted) their statistical productions to interna- 
tional requirements. 


Describing the instruments, procedures, and practices of data col- 
lection (questionnaires, manuals, response rate), data treatment 
(databases, metadata, methods of checking data “quality”), and 
data production, publication, and presentation (use of inferential 
statistics, estimates, figures, tables . . .); as well as the weight in the 
statistical decisions of managers, subject experts, statisticians, and 
computer programmers. 
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f) Investigating the biographies, profiles, networks, discourses, 
interests, and positions of the actors—NGOs and government 
members, IO officials, experts, trade unionists, and so on— 
participating in the quantification process (personal files, corre- 
spondence, interviews). 


g) Considering other data producers and users in order to under- 
stand whether an IO quantification is “influential” and/or 
“influenced.” 


h) Linking the studied IO statistical “systems” to specific “political 
configurations” by analyzing the consistency between IO policies 
and norms and IO data analysis, presentation, and use. 


What Challenges? 


Quantification studies involve several challenges. First, the researcher's 
understanding of data production and analysis are essential for identify- 
ing the technical issues and choices at stake. Knowledge of the functioning 
of institutions and the subject in question (health, labor, etc.) is also use- 
ful. This is all the more important in the case of international quantifica- 
tion studies, since they also require an understanding of multiple different 
national contexts. Researchers therefore need to adopt a genuinely interdis- 
ciplinary approach. 

Second, and from a more theoretical perspective, quantification stud- 
ies often show a tendency toward functionalism. Indeed, statistics are fre- 
quently viewed as being produced for essentially political purposes—for 
instance, to maintain some kind of political advantage (e.g., North over 
South) or to push a particular perception of a question (e.g., climate change, 
the efficiency of private education). However the why dimension cannot be 
reduced to a single (or simple) political purpose(s) for the data produced; in 
most cases, statistics are coconstructed and allow for different—sometimes 
contradictory—uses, political or not. 

Third, quantification studies may sometimes draw on a rigid relativism. 
If statistics are viewed merely as socially constructed artifacts, their capacity 
to produce knowledge can be missed. Indeed, the act of measuring phenom- 
ena can also be legitimately considered a form of raising relevant questions: 
for instance, if the quantification of climate change may certainly support 
specific political interests, it can also describe “tangible” environmental and 
human experiences. 
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Finally, quantification studies may also tend to stereotype the “statistical 
systems,” when they are often hybrid and/or challenged by unforeseen uses. 
As regards hybridity, while the “neoliberal” World Bank uses micro-data 
to measure local loan efficacy, it still relies on macro-statistics for assessing 
national debts. Similarly, concerning unexpected data uses, the European 
Union certainly prioritized benchmarks on employment rates, reducing 
unemployment visibility, but this choice also resulted in an increased focus 
on precarious wage earners. 


To Go Further 


Beerli, Monique J. 2017. “The Power to Count and the Stakes of Counting: An Inqui- 
ry into the Quantified Production of Humanitarian Insecurity.” Global Governance 
23 (1): 57-70. 

Davis, Kevin E., Benedict Kingsbury, and Sally Engle Merry. 2012. “Indicators as a 
Technology of Global Governance.” Law & Society Review 46 (1): 71-104. 

Hansen, Hans Krause, and Tony Porter. (2017). “What Do Big Data Do in Global 
Governance?” Global Governance 23 (1): 31-42. 
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BOX S 


Analyzing Charts, Infographics, and Dataviz 


Benoit Martin 


“Charts,” “infographics,” and “dataviz” are several words that express a single 
medium, namely pictures that show data. Influenced both by fluctuating 
trends and technological improvements, dataviz gathers heterogeneous fig- 
ures, from austere statistical graphs to colored figurative infographics. This 
is nothing new, however, since some of these efficient pictures were drawn 
from centuries ago (Tufte 1981). 


Omnipresence of Dataviz in IO Expertise 


“Pictures made out of data” are omnipresent in the products of IO exper- 
tise (reports, studies, manuals) and thus in its resulting communication 
(news, tweets, slides). Concretely, IOs usually produce dataviz in order to 
a) expose conceptual thinking (a diagram explaining Human Development 
Index composition); b) illustrate empirical cases (a curve showing poverty 
rate reduction); or c) evaluate global situations (a dashboard following Sus- 
tainable Development Goals indicators). Whatever the function, dataviz 
strengthens the scientific character of analyses by combining the evidence 
of statistics with the power of pictures. In a thick global report, charts often 
provide the first visual landmarks the reader’s eye is drawn to and thus 
remembers. Likewise figurative infographics in tweets (proportional woman 
pictograms to talk about femicide) have a greater impact than written ones. 

IOs are far from being monolithic actors. Their dataviz production often 
involves various specialists (experts, statisticians, illustrators, designers, etc.), 
international staff and/or member states representatives who work together, 
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more or less independently, while IOs sometimes outsource dataviz produc- 
tion. The resulting dataviz picture depends on the context of production. 
Pictures can reveal clues of the authors’ backgrounds (and biases) especially 
when paying attention to the balance between abstraction and figuration, 
functionality and aesthetic, multi- and unidimensionality, or even familiar- 
ity and originality (Cairo 2012). Dataviz products then circulate and are 
(re)appropriated with potentially contrasting interpretation, which scholars 


Analyzing Charts, Infographics, and Dataviz 


should pay attention to. 


Layers of Subjectivity 


The production process of dataviz implies choices that frame the final 


discourse: 


l. 


Obvious but often overlooked, by deciding to transform informa- 
tion into dataviz, the authors choose to highlight a specific issue. 


This issue holds several dimensions (equivalent to columns in the 
dataset). The example of migrations is telling. We could either 
choose raw numbers of migrants (flows, balances) or indirect 
calculated ratios (share of total, trend over time) or qualitative 
aspects (status, gender, nationality). 


Each type of dataviz conveys a specific argument. In its FT Visual 
Vocabulary table, the Financial Times visual and data journal- 
ism team distinguishes core functions through ad hoc designs: 
deviation, correlation, ranking, distribution, change over time, 
magnitude, part-to-whole, spatial, and flow. Picking one is not 
automatic nor exclusive. Thus a dataset can fulfill various func- 
tions and dataviz can bring complexity rather than simplicity or 


clarity. 


Ultimate detail parameters, whether related to statistical content 
(scale, period, axis) or aesthetic shape (size, colors, layout) influ- 
ence the final interpretation. The plethora of popular scientific 
books investigating “how to lie with charts” attests to the mallea- 
bility of these choices and their—not always—desired outcomes. 
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CHAPTER 13 


Budget Analysis 


Svanhildur Thorvaldsdottir and Ronny Patz 


Many questions can be asked when studying the budgets of international 
organizations (IOs): which IO has the biggest budget? Which IO has the 
fastest growing budget? Where do IOs spend money, and what policy 
domains are underfunded? Does the United States dominate the IOs it 
contributes to? How much influence do nonstate donors such as the Gates 
Foundation have on IOs? Or, who decides where IOs spend their money? 
Answering these questions brings a number of challenges, for example, how 
to measure the size of budgets, revenues, and expenditures while achieving 
validity, reliability, and completeness. 


What? 


Measuring and analyzing an IO budget can mean many things. At the 
national level, a budget usually refers to legislation adopted to fix annual or 
multiannual spending, based on income generated from taxes, debts, and 
other revenue sources of the state. For IOs such as the European Union 
(Becker et al. 2018), United Nations system agencies (Patz and Goetz 2019; 
Thorvaldsdottir 2017), or regional IOs in the Global South (Engel and 
Mattheis 2019), however, a budget can have many different meanings. For 
example, the annual EU budget and procedures compare well to a national 
budget, while the IOM budget is largely based on expected income for indi- 
vidual projects instead of centrally adopted spending priorities. 

Depending on the IO, the regular budget represents broader or narrower 
spending categories. Sometimes member states give more autonomy to the 


186 


Budget Analysis 187 


IO bureaucracy to allocate regular funds according to political priorities, but 
sometimes they fix spending down to individual staff positions. And not all 
budget elements are regular, as many IOs depend on additional voluntary 
funding—mostly earmarked by government donors for specific purposes— 
that may not appear in the regular budget. Some IOs formulate aspirational 
or “integrated budgets” through which they signal spending needs, and then 
launch fundraising appeals to fill their budgets with voluntary contributions 
from governments, foundations, corporations, individuals, and other donors 
(examples: UNHCR, WHO). In sum, budget figures can mean different 
things for each IO: in some, they represent actual spending; in others, they 
represent fundraising goals; in still others, they are a combination of both. 

Measuring budget figures also requires careful consideration of the differ- 
ence between budgets, revenues, and expenditures. A budget is the political or 
administrative decision about political priorities often represented in budget 
lines linked to certain policy targets. Revenue represents actual income an 
IO generates, through member state contributions in most IOs, through 
intellectual property services in WIPO, or through its own resources in the 
EU. Expenditures represent IO activities, those planned but also those not 
planned. Revenues and expenditures may also be mere contractual obligations, 
like budget commitments (expenditures that may be incurred) or pledges 
(revenues promised) or actual disbursements, such as bills paid or donations 
actually received. This distinction is important: UNESCO adopted an offi- 
cial budget that included the United States’ financial obligations for several 
years after the country halted its contributions in 2011, but it also adopted 
a separate implementing plan reflecting the reality of major cuts required. 

As in the case of UNESCO, state contributions are usually IOs’ main 
sources of revenue. However, some IOs, like the EU, may have their own 
resources (see Becker et al. 2018), and others, like UNICEF, rely heavily 
on individual and private donations. Similarly, philanthropic and corporate 
contributions to IOs have existed at least since the League of Nations but 
are becoming more important recently (Seitz and Martens 2017). Including 
these alternative revenue sources is crucial for research designs that focus on 
the limitations to state influence or on bureaucratic autonomy in IOs. 


Why? 


Understanding IO financing matters because IOs provide crucial global 
public goods and depend on resources to do so. They are also significant 
distributors of multilateral and sometimes even bilateral aid (“multi-bi aid”), 
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in some cases comparable to medium-size states, making them central play- 
ers in (sustainable) development and humanitarian aid. Furthermore, IOs 
are not mere tools of member states. Where IOs have access to sufficient 
and flexible resources, they can gain certain levels of autonomy, giving them 
independent influence and power (Ege and Bauer 2017). Thus understand- 
ing their financing can further our collective understanding of their overall 
role in global governance. 

Additionally, analysis of IO budget figures can help trace the influence— 
formal or informal—of various actors external to the IO itself. For example, 
through tracing influence of government donors (Thorvaldsdottir 2017), 
looking at the scope of private contributions (Seitz and Martens 2017), or 
examining broader questions of “trojan multilateralism” (Sridhar and Woods 
2013), budget data can make observable some of the mechanisms through 
which these actors wield influence. Given how much of the politics of IOs 
happens informally or behind closed doors, sometimes financial data are the 
only documentary evidence for a systematic analysis of influence. 

IO budgets, revenue, and expenditures are also among the most mea- 
surable dimensions of IO activities. They provide an overview of the size 
and growth of IOs, as well as their geographical or policy foci. Carefully 
constructed, budgetary data also permit comparison across IOs and can 
provide reliable dependent or independent variables, especially in quantita- 
tive research (see, e.g., Thorvaldsdottir, Patz, and Goetz 2022; Lall 2020; 
Eichenauer and Reinsberg 2017; Humphrey and Michaelowa 2019 for 
recent work using a variety of financial variables). 


How? 


Various ways exist for measuring IO budgets, revenue, and expenditures. For 
more recent periods, myriad datasets exist with varying levels of detail, accu- 
racy, and coverage. Common, but not necessarily reliable and valid, sources 
are the Yearbook of International Organizations' (includes annual budget 
figures but without clear-cut budget definitions or expenditure data); the 
OECD-DAC databases’ (not always comprehensive and limited time cover- 
age); the financial statistics provided by UNSCEB for the UN system? (with 
consistency issues going back in time); and individual IOs’ websites with 


1. https://uia.org/yearbook 
2. https://stats.oecd.org/Index.aspx? DataSetCode=MULTISYSTEM 
3. https://www.unsceb.org/content/un-system-financial-statistics 
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downloadable budget sheets (not necessarily consistent over time). Recently, 
the International Aid Transparency Initiative (LIATI) has started providing 
additional data, especially at the project level‘ but also by agency.’ 

Often the level of detail in these datasets is insufficient or definitional 
terms (i.e., what exactly is included in the numbers provided) are unclear, 
rendering primary research of official documents necessary. The most reli- 
able and detailed documents for this purpose are audited financial state- 
ments. Most IOs produce these, and they can usually be accessed directly 
from IOs or from relevant national ministries. Annual reports can also be 
useful but not all include detailed financial data. However, they often pro- 
vide more contextual information than audited financial statements, which 
can help improve validity. When working with preexisting databases, it is 
good practice to check them against official documents to validate both 
accuracy and measurement, and to ensure that the data appropriately reflect 
the research question posed. 

Understanding what is included in IO budget documents and datasets 
at different points in time is key for validity of time series, panel, and cross- 
sectional analyses. For example, studying EU finances may require different 
budget data for different questions: data on commitment appropriations in 
the EU budget can answer questions about general political priorities, while 
payment appropriations determine how much member states have to pay into 
the budget and thus why net contributors care about these in particular. 
Measuring EU budget outturn, namely actual expenditures, is more inter- 
esting for research on budget implementation or on the influence of the 
European Commission and of national bureaucracies on which beneficiaries 
ultimately receive EU funds. 

‘These examples from the EU are mirrored in other IOs. They show that 
financial and budget data collection requires fairly detailed expertise on the 
finances of an organization. Researchers must invest considerable time to 
ensure that consistent—and thus comparable—budget, revenue, and expen- 
diture data are collected. Among these challenges is appropriately defining 
boundaries of inclusion. Many IOs have extensive “budgetary galaxies” 
(Crowe 2017), including segmented side-budgets (Patz and Goetz 2019), 
trust funds (Reinsberg et al. 2015), and various earmarked voluntary contri- 
butions both inside and outside core budgets (Graham 2017). Inconsistent 
decisions on which of these to include lead to distortions in cross-sectional 
and longitudinal studies. Different types of budget data may also be rela- 


4. http://d-portal.org/ctrack.html#view=search 
5. https://www.iatiregistry.org/publisher 
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beled or modified as international accounting standards change (Bergmann 
and Fuchs 2017). 

Next, researchers should be aware that budget figures are usually reported 
in nominal terms, not real. A nominal budget increase between years can 
actually translate into a real budget cut once inflation is taken into account. 
A lot of political debates in IO governing bodies revolve around whether 
budget growth should be real or nominal. A similar phenomenon exists on 
the income side: nominal increases (or decreases) in a donor’s contributions 
in the currency of an IO (Euro, US dollar, Swiss franc, etc.) can be due to 
exchange rate fluctuations rather than real shifts in donor preferences. Ignor- 
ing inflation or exchange rates may lead to misinterpretation of the causes 
for shifts in IO income or expenditures. 

Finally, researchers need to consider the level of detail in the budget data 
they want to study since this can materially shape findings: macro-level 
spending categories in an IO budget may shift in name but remain very sta- 
ble over time. This perceived stability may mask punctuated or even erratic 
responses to outside influence or actual needs. For example, WHO may 
have the same overall expenditures in the Democratic Republic of Congo 
in two consecutive years, but in the first year the allocation comes from the 
core budget and is spent on health-system support while, in the second year 
it is based on earmarked contributions from a special fund to tackle Ebola. 
Thus while country-level financial data may suggest stability, policy-level 
data can reveal significant shifts in focus of the IO. 


What Challenges? 


‘The various definitions of what a budget is in different IOs is the starting 
point for much confusion. Many researchers do not clarify which aspect 
of IO budgets—revenues, expenditures, projects, and so on—they study, 
which can obscure what the data actually measures and the extent to which 
different studies are comparable. 

The central challenge, however, regards the availability, quality, and 
comparability of budget data.° Constructing a dataset with reliable finan- 
cial data going past the last decade or so can be fraught with difficulties. 
While websites of IOs have improved in recent years, with some even mak- 
ing downloadable financial datasets available, finding official figures can be 
hard unless there exists a dedicated and up-to-date IO budget website. 


6. For the UN system see Dag Hammarskjöld Foundation and UN MPTF Office (2018, 
chapter 3). 
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One difficulty is simply locating final and authoritative figures. For the 
UN core budget, the official final figures are decided in a GA resolution. 
However, complete budget and staff figures as well as political priorities are 
only found in the draft budget documents. Nowhere are both consolidated 
into a single public document. In the case of UNHCR, detailed donor 
contributions are published in audited financial statements while detailed 
expenditures at headquarters or field level are inconsistently reported over 
time. Some official expenditure databases include only certain expenditure 
types, such as project spending, but not all other institutional spending. 
Contributions from the same donors but to different funds of an IO may be 
reported in multiple locations; private donations may come from “private” 
foundations closely linked to governments but are not reported as state sup- 
port; and other nonstate revenue may only be recorded in aggregate formats, 
limiting more fine-grained analyses. 

Finding historical data online may often be impossible. Experience with 
the UN system furthermore shows that archives may not collect budget- 
ary or financial documents in a consistent manner, so field research may 
not yield complete documentation either. Older financial documents may 
sometimes be unreadable or require extensive manual labor to transform 
into machine-readable data. Sometimes legible financial documents are only 
available in language versions that researchers do not speak. 

Last, comparing data across IOs may lead to validity issues due to so-called 
double delegation (Michaelowa et al. 2019). For example, the EU heavily cofi- 
nances regional organizations in the Global South (Engel and Mattheis 2019) 
and UN organizations (Michaelowa et al. 2019). Some UN funds and agen- 
cies provide significant funding for other UN agencies. This double delegation 
can lead to double counting, which may be problematic when assessing overall 
multilateral spending, or when comparing revenues of two IOs where one is 
also a major donor to the latter (e.g., the EU for the AU). 

Pragmatic solutions to the challenges of IO budget research will often 
include developing nuanced understanding of what data are available and 
adapting research questions accordingly. Being transparent about definitions 
and data sources is paramount, so that IO research including financial data 
can slowly accumulate insights, even if researchers study IO budgets with 
varying questions and at different levels of analysis. 


To Go Further 


Goetz, Klaus H., and Ronny Patz. 2017. “Resourcing International Organizations: 
Resource Diversification, Organizational Differentiation, and Administrative 
Governance.” Global Policy 8 (S5): 5-8. 
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Patz, Ronny, and Klaus H. Goetz. 2019. Managing Money and Discord in the UN: 
Budgeting and Bureaucracy. Oxford: Oxford University Press. 
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INTERLUDE III 


What lOs Talk About When They Talk about 
Themselves, and How They Do It 


Davide Rodogno 


This interlude, the title of which is a clin d'veil to a famous book by Haruki 
Murakami (2008), reflects on the ways international organizations (IOs) 
talk about and to themselves and to us (i.e., lay persons, students, scholars). 
What do they tell us? How and why? What do they communicate and what 
do they choose not to communicate? How do technologies favor and ham- 
per, free and constrain the ways in which IOs talk? This set of questions is an 
invitation to think about these strange creatures. 

In their contributions, the authors reflect on the methods to study a 
variety of products, outputs and outlets imagined, set up and used by IOs. 
Their analyses are of critical importance because they draw our attention to 
the ways IOs talk to us, we the people, we the recipients of their talk and of 
their actions. From my perspective, namely the perspective of a scholar of 
international history and politics, the communication of IOs is more than 
just a simple activity, rather it is an existential one that—among others— 
proves their vitality both to these organizations internally as well as to the 
outside world. The act of communicating is for them, as for other institu- 
tions or living species, a sign of life. Put differently, for IOs Communico ergo 
sum. In this short interlude I deliberately and alternatively use the verbs to 
talk and to communicate referring on the one hand to speak in order to give 
information, such as expressing ideas including feelings, and on the other 
hand to communicate by spoken as well as unspoken words such as images, 
photographs, artifacts, and data. I include architecture or logos (see box 
n—Branding Analysis) into the concept of communication (Bak McKenna 
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2021). To talk might also be extended to negotiations (as in peace talks) and 
refer to informal and formal discussions in so far as they indicate a specific 
way in which a given organization expresses itself in a specific way to take 
action. 


Different Talks—Origins, Legal Documents, Situated Knowledge 


To begin with, it is important to bear in mind that IOs might display several 
concomitant talks using slightly or significantly different ways of expressing 
themselves. Therefore the multiplicity of talks on the same issue is some- 
thing that must be considered when researching these creatures. I refer to a 
given IO talking about the same issue in various ways as well as different IOs 
talking about a given issue in very different ways. As an example of the for- 
mer case, we can think about the International Committee of the Red Cross 
talking (negotiating) very discretely for access to victims referring to interna- 
tional humanitarian law and—at the same time—firmly, vigorously talking 
(advocating) with the warring parties in that same conflict on the interest 
they have in knowing and applying international humanitarian law. As an 
example of the second case, we can think about the African Union and the 
United Nations and the ways in which they talked about events in Darfur 
throughout the 2000s (Mandani 2009). They used (or not) the term geno- 
cide and kept proposing different talks. The multiplicity of talks should not 
be hastily dismissed as evidence of inconsistency, incoherence of duplicitous 
behavior because it informs the remarkable ability of these organizations to 
adapt, which we—scholars—should not downplay or, worse, ignore. From 
a methodological point of view, it is also worth encompassing the analysis of 
a given organization's perceptions (and misperceptions) of the recipients of 
their talk and how they imagine their audiences. 

When examining the ways in which IOs communicate, scholarly analy- 
ses should pay attention to the fact that each communication is the result 
of several layers of mediation. Each is filtered and often the result of com- 
promises. Mediations are determined by the object of the communication, 
by its inherent and perceived stakes, and the institutional culture(s). They 
are mediated by the individuals involved with it and the specific historical 
context as well as by the format of the talk. Some of these ideas might entail 
a normative intention, while other ideas might have a prescriptive, descrip- 
tive, or informative nature (see box r—Studying Ideas). Normative ideas 
might be translated into international law, the language the international 
community decides to share (see chapter 7—Legal Research). 
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Among other things, IOs are social spaces and loci where knowledge is 
created or at least places where knowledge is shared, used, and communi- 
cated (Kott 2017). IOs might have an interest in preserving, guarding, and 
if possible expanding their role as knowledge-keepers, knowledge-makers, 
and knowledge-spreaders. Before being communicated externally, this 
knowledge is discussed internally. Knowledge creation and its circulation 
are processes of cocreation. These processes are not necessarily or systemati- 
cally democratic. Cocreation does not presuppose equity or equality; and 
we know that power relations are omnipresent in influencing and shaping 
knowledge. For instance, the League of Nations was designed by imperial 
powers for imperial powers and their own interests and knowledge (Miil- 
ler 2020). The discrepancy observed within the organization between self- 
determination talks and the reality of the organization’s support for oppres- 
sive rule must be acknowledged. The ways in which a student imagines the 
mandate system through which the League of Nations was, for the first time, 
formally involved in the ruling of human societies might be a good example 
of a (dead) IO’s talk. A student might assume that the talk as progressive and 
emancipatory, geared toward the independence of all mandated territories, 
since this was the official discourse and the rhetoric the League of Nations 
put forward. On the contrary, they might examine the mandate system as 
entailing a talk that was progressive and a reality, which remained oppres- 
sive. Methodologically, the consequences are significant. One may either 
posit that the Permanent Mandate Commission of the League of Nations 
was the expression of a revolutionary entity working for the emancipation 
of colonized populations while another may claim that colonial—hence 
oppressive—mandates were supposed to have an emancipatory and civili- 
zational role with mandatory powers using their authority and right to rule 
on the basis of an allegedly superior civilization. In other words, despite the 
scientific value of analyzing IO official talks, we should acknowledge that 
these do not necessarily reflect IO action. 

Another methodological concern relates to the importance of situating 
knowledge and the subsequent talk of IOs. For instance, after 1945 and for 
several decades, the United Nations development projects were based upon 
assumptions and knowledge that came from ex-colonial administrators who 
started new careers as international civil servants. These visions combined 
with a situated, specific kind of knowledge and were circulated and com- 
municated within the organizations. These development programs were thus 
enforced by the United Nations and—some—of its agencies based on a 
certain kind of knowledge with certain ideological visions. The technical 
nature, the alleged benevolent nature of the action, the seemingly apoliti- 
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cal talk (Louis and Maertens 2021) concealed a different reality, one that 
perpetuated colonial policies in a postcolonial world. The methodological 
take away point is identical: no matter how theoretical the study of IOs is, a 
precise and granular knowledge of the context, of the history and politics is 
indispensable when we examine how and why IOs talk to us. 

IOs are loci, physical, and virtual; but they are much more than merely 
places. They are agents in that they have agency. What complicates research 
is understanding the specific contribution, the inputs that IOs give in a 
given context, before they talk, when they talk, and after the talk has hap- 
pened. For instance, Heymann (2020) showed that perceptions of climate 
challenges have changed significantly: regional climate issues and problems 
of arid regions have received less attention than global climate changes. 
Persistent misconceptions and a lack of understanding of arid zones are 
rooted in misguided colonial ideologies and expertise, propagated by United 
Nations initiatives such as the United Nations Educational, Scientific and 
Cultural Organization Arid Zone Programme. This is a typical and certainly 
not exceptional example of the—biased—input an IO can give on a specific 
issue on which it claims to have the authority and the legitimacy to talk and 
to produce knowledge from. 

We know that the official talk of IOs goes public after several filters and 
multiple compromises among stakeholders—whomever they might be— 
have taken place. This in turn explains the jargon of some IOs. An official 
written or oral communication or the utilization of a single photograph 
might follow lengthy and thorough reflections and negotiations. This may 
also explain the overabundance of statistical data (and data in general) in 
IOs’ official communications. Data represents an attempt of objectivity 
through the allegedly impartial and universal language of science and num- 
bers (see chapter 12—Statistics and Quantification). Data and figures have 
also a symbolic potential, are given an untold symbolic and in some cases 
religious dimension that help IOs depoliticize and neutralize some issues 
or on the contrary politically overcharge IO communications (see chapter 
11—Discourse Analysis, box m—Analyzing Maps, box n—Branding Analysis, 
box s—Analyzing Charts, Infographics, and Dataviz). For instance, data takes 
the center stage of annual reports. The introduction of these documents 
often starts with an avalanche of data that overwhelms readers. They are 
there to legitimate and prove the authority of the talk. At the same time— 
and this has been the case for more than one hundred years—IO reports 
contain images, photographs, and videos (or animations for Internet out- 
puts) injecting emotions and a further—different and complementary— 
register, which often ambitions to humanize the talk (see chapter 9—Visual 
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Methods, chapter 10—Document Analysis: A Praxiographic Approach, and 
chapter 11—Discourse Analysis, box |—Visual Archives, box o—Artifact 
Analysis, box p—Semiology of Websites, and box q—Analyzing Tweets). If for 
some organizations the use of images was (and to some extent still is) a 
simple afterthought or a decoration, other organizations have reflected on 
the role of images as a powerful way to communicate or convey their politi- 
cal and/or ideological stances. Images serve specific and multiple purposes 
and are part of sophisticated strategies, an integral part of the ways in which 
IOs express themselves. 


Taking Context into Account When Researching Documents 


Practices of production and circulation of knowledge and its communication 
are specific to each IO but they are always determined by historical, politi- 
cal, and cultural contexts. For instance, the influence of Protestant Chris- 
tian charity notions and Calvinist heritage shaped the way the International 
Committee of the Red Cross (ICRC) was designed, operated, and talked (or 
not), and even the way it stored its archives. Like many other organizations, 
the ICRC has a hard time admitting its mistakes and its archives are full 
of meaningful silences that scholars must interrogate and investigate. IOs’ 
silences, especially for institutions for whom communication is an existen- 
tial need, deserve more scholarly attention. Moreover, the ways in which IOs 
were founded seventy or one hundred years ago are interesting insofar as 
there are plenty of analyses scholars can offer as to their enthusiasm or reluc- 
tance to embrace new media over time from photography to radio, from cin- 
ema to television and the Internet. For instance, an archeology of a given IO 
website since its inception might invite original reflections on the changes 
that IO underwent over the last thirty years (see box o—Artifact Analysis). 
Scholars contributing to this part of the handbook draw our attention 
to three issues when it comes to the need of careful contextualization. First, 
whether we are researchers or informed readers we should not assume that 
international law and international organizations are inherently good. As 
Hurd puts it, this premise is pervasive in global governance scholarship 
despite its obvious empirical and political pathologies. It impedes empiri- 
cal research, but luckily it is easy to avoid. An effective way to do so is to 
investigate the mediated, filtered nature of IO communication. Second, the 
connection between communication and a given IO’s agency and intention- 
ality should be kept in mind by scholars at all times. Hence identifying the 
individual (or unit, department) that communicates is important and chal- 
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lenging because of the collective authorship as well as exogenous and endog- 
enous factors, including self-censorship. I warmly invite researchers to adopt 
a multimethod research design that might help understanding how the talk 
is imagined before being talked. Distances and discrepancies between the 
imagined talk at its inception and its actual delivery are as relevant as the 
measurement of the impact of the talk. Third, our (i.e., scholars) own posi- 
tionality should be openly and fully disclosed from the outset. 

I would like to conclude this appeal for further attention to the history 
and politics of IOs drawing the attention to artifacts (see box o—Artifact 
Analysis) and architecture as insightful ways to study how IOs talk. Research 
conducted solely online might miss the material dimensions inherent to any 
individual IO. Last but not least, even when archives exist and are avail- 
able, and even when the scholar can afford traveling to these archives it is 
worth insisting on the importance of cross-referencing them. None of these 
archives can be enough to understand what, how, and why IOs operate. 
They must be combined with other primary and secondary sources (see part 
5—Combining). 

Scholars have already noted the neglect of the cultural dimension of IO 
public communication (Ecker-Ehrhardt 2018). I would add the historical 
dimension to the cultural one as an area where further research is needed. 
When it comes to IO communication, a final conclusive invitation I have 
is to study more carefully and thoroughly the distance that exists between 
how IOs have imagined and currently imagine their audiences and how the 
recipients receive, interpret, and critique that information. New research on 
IOs’ Instagram and Twitter accounts (see box p—Semiology of Websites) and 
the specific talk of these organizations on these and other platforms might 
reveal interesting aspects of the abilities and skills these organizations have 
to adapt to new technologies and to reach out to us, the people (Corneliu and 
Zaiotti 2021). 
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PART 4 


Measuring 


Introduction 
Measuring 


Fanny Badache, Leah R. Kimber, and Lucile Maertens 


While the previous part, Documenting, presented and discussed methods to 
research and understand what international organizations (IOs) produce, 
such as archives, documents, discourses, statistics, and visual artifacts, this 
section turns to the use of numbers and quantitative data in the research on 
IOs. We present how researchers can rely on multiple methods to measure 
different dimensions of multilateralism. 

In social sciences, scholars have to a large extent relied on numbers to 
describe and understand social phenomena. IO studies are no exception (see 
interlude 1V—Challenging IOs through Numbers). Historically, quantitative 
data have been used to understand states’ behavior within IOs, and in par- 
ticular voting behavior (see chapter 14— Voting Analysis). Scholars have also 
developed quantitative tools to count the number and type of IOs. Thanks 
to these research efforts, IO scholars can now use a variety of databases 
on IOs that provide characteristics such as membership and institutional 
design. More recently, scholars have started using quantitative information 
to study other sets of actors aside from states, such as IO staff and leader- 
ship (see box t—Navigating Human Resource Statistics and box u—Building 
Databases on Individuals) and civil society organizations. In addition, quan- 
titative methods are increasingly used to research IO documents such as 
recorded speeches and reports (see chapter 17—Computerized Text Analysis). 

Quantitative methods allow scholars to achieve various research objec- 
tives. First and foremost, with numbers, scholars can provide a systematic 
description of phenomena over time and across many units. Numbers 
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enable researchers to highlight trends and carry out comparisons (see chap- 
ter 15—Statistical Analyses with IO Data, and chapter 16—Large-N Data 
and Quantitative Analyses). Second, large-N methods can deepen our under- 
standing of causal relations by looking at relationships between variables 
across many cases. Although causal inference can be achieved based on 
both qualitative and quantitative approaches, statistical methods examine 
the relation between two variables (or more) in a covariational perspective. 
Third, numbers enable researchers to seize interactions between and among 
actors in a systematic way (see chapter 17—Computerized Text Analysis, 
chapter 18—Multiple Correspondence Analysis, and chapter 19—Social Net- 
work Analysis). The contributions present methods able to objectify the rela- 
tionships and links between different entities in a way that can be insightful 
to answer research questions on power dynamics and cooperation between 
IO actors, for instance. 

In this part entitled Measuring, the reader will find two types of contri- 
butions. The first one presents why and how researchers use quantitative 
data—either produced by researchers themselves or using data emanating 
from IOs (see chapter 14—Voting Analysis, chapter 15— Statistical Analyses 
with IO Data, and box t—Navigating Human Resource Statistics). Contribu- 
tions shed light on the potential but also the challenges related to the use 
of IO statistics, for instance, in terms of access, comparability, and validity. 
‘The second type of contributions focuses on quantitative methods of data 
analysis (see chapter 16—Large-N Data and Quantitative Analyses, chap- 
ter 17—Computerized Text Analysis, chapter 18—Multiple Correspondence 
Analysis, and chapter 19—Social Network Analysis). In addition to strict sta- 
tistical analyses, scholars also mobilize quantitative methods to examine and 
present data that may have been collected thanks to other methods such as 
interviews or document analysis to name a few. This acknowledgment comes 
as a kind reminder that quantitative and qualitative methods should not 
be put face to face, in an antagonistic perspective, but rather integrated to 
best answer our research questions (interlude IV— Challenging IOs through 
Numbers). 


CHAPTER 14 


Voting Analysis 


Erik Voeten 


International organizations (IOs) often resolve contentious issues by vot- 
ing. This chapter first discusses the methodological challenges involved in 
translating observed votes into measures of theoretically relevant concepts. 
It then examines how institutional and strategic context shapes how we ana- 
lyze votes. 


What? 


International organizations sometimes resolve contentious issues by vot- 
ing. To the extent that these votes are public, they provide useful data for 
scholars about state positions on controversial issues. For example, scholars 
have analyzed United Nations General Assembly voting to examine how 
a state positioned itself in the Cold War East-West ideological conflict or, 
more recently, a state's acceptance of the US-led international order (Voeten 
2000). Scholars have analyzed the European Parliament to understand 
national and partisan contestation over left-right socioeconomic conflict 
and European integration (Hix, Noury, and Roland 2006). In the European 
Court of Human Rights, scholars have analyzed judicial voting behavior to 
examine which judges favor a more activist or more restrained court (Voeten 
2007). Such analyses are only possible in international organizations where 
voting is both public and relatively common. For example, Court of Justice 
of the European Union minority are not public. The United Nations Secu- 
rity Council has very few contentious votes. Thus voting analysis applies 
only to a select group of international organizations. 
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Why? 


Analysts study voting in IOs for two distinct purposes. First, some scholars 
are interested in better understanding organizational decision-making. A 
vote, in this context, is of interest because it carries policy, legal, or political 
consequences. So scholars may ask whether foreign aid or other economic 
considerations shape UN Security Council (UNSC) decisions (Vreeland and 
Dreher 2014). Or how public opinion shapes government vote choices in 
the Council of the European Union (Hagemann, Hobolt, and Wratil 2017). 
Second, scholars use votes to construct measures for the ideologies of 
states, judges, or other actors. UN General Assembly (UNGA) votes are a 
popular data source not because scholars believe that its nonbinding reso- 
lutions are important but because UNGA votes reveal policy positions of 
many states over a wide range of global issues at regular intervals over a 
long period. Measures of UN voting correlate with numerous consequential 
outcomes that have little to do with the UNGA, including the likelihood 
of interstate militarized disputes (Gartzke 1998), the distribution of foreign 
aid (Alesina and Dollar 2000), and the lending behavior of the World Bank 
and IMF (Thacker 1999). Others use these measures as dependent variables, 
for example studying whether domestic leadership changes result in foreign 
policy changes (Mattes, Leeds, and Carroll 2015) or whether trade with 
China affects foreign policy interests (Flores-Macias and Kreps 2013). 


How? 


Empirical spatial models simultaneously estimate ideal points and roll-call 
parameters from observed vote choices. There is a vast literature explain- 
ing these models, even though most of it focuses on legislatures and courts 
rather than IOs.' There are also numerous options to estimate these models. 
This ranges from traditional ready-made models that can be estimated in 
R or Stata, such as W-NOMINATE (Poole et al. 2008), to customizable 
Bayesian frameworks (Grant et al. 2017). 

Analysts need to make several methodological choices. First, abstentions 
have different meanings dependent on context. In the UN General Assem- 
bly, abstentions are extremely common and have long been understood as a 
less strong signal of disapproval than a no vote (Lijphart 1963). This requires 
an ordinal model, which is not common in the broader literature, although 
adjustments are reasonably straightforward (Voeten 2004). 


1. A good introduction is: (Poole 2005). 
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Second, analysts must decide how to model dynamics. One particularly 
troubling issue is that changes in voting agreement may reflect changes in 
the agenda as well as changes in foreign policy preferences. For instance, a 
year with many votes on the Israeli occupation of Palestine will affect vot- 
ing agreement scores even if no state changes its preferences. A dynamic 
model separates shifts in the UN’s agenda from shifts in ideological positions 
by holding constant the roll-call parameters of identical resolutions (Bailey, 
Strezhnev, and Voeten 2017). For example, in 2016 US president Barack 
Obama decided to abstain rather than vote against a long-standing UN 
resolution on the US embargo on Cuba. In 2017, newly elected president 
Donald Trump reversed the UN vote. Because we know that the resolutions 
are identical, we can fix the roll-call parameters across these years, which 
helps us identify preference change. These dynamic estimates correlate well 
with known changes in foreign policy ideology, such as changes in domestic 
levels of democracy and capitalism, as well as domestic left-right switches in 
government (Voeten 2021). 

Third, analysts have to decide on dimensionality. In the context of the 
UN General Assembly, Michael Bailey and I examined whether there is a 
stable, important, and interpretable second dimension (Bailey and Voeten 
2018). We found that from the mid-1960s to the mid-1980s, North-South 
conflict constitutes a stable second dimension. In the periods before and 
after, the second dimension neither is stable nor easily interpretable, though 
it is sometimes important. In most applications, one-dimensional estimates 
have conceptual advantages with minimal losses in explanatory value. Yet 
analysts who care about particular issues may well consider estimating ideal 
point models on subsamples of the data.* For example, voting behavior on 
nuclear issues often poorly fits the first dimension. 

Finally, IR scholars are often interested in how often two states vote 
together? Sometimes they use ideal point distances to estimate the degree to 
which states have “shared foreign policy interests.” This interpretation can be 
problematic. Votes are not relational. States vote on resolutions on human 
rights, the Middle East, nuclear weapons, etcetera. The patterns underly- 
ing vote choices reflect ideological positions not shared interests with any 
specific country. 

To illustrate this, consider the triad Saudi Arabia, Iran, and the United 
States. Saudi Arabia and Iran often vote together even though they are 
adversaries. The United States and Saudi Arabia rarely vote together even 
though they are strategic partners. This makes sense in terms of ideologies 


2. For code and examples, see: https://github.com/evoeten/United-Nations-General-Asse 
mbly-Votes-and-Ideal-Points 
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over global issues. Saudi Arabia and the United States have common strate- 
gic interests. But they have profoundly different views on global issues, such 
as human rights. By contrast, Iran and Saudi Arabia share similar views on 
those issues even as they are regional rivals. Thus ideal point differences are 
estimates of how different two countries’ preferred resolution of global issues 
is, which may differ from the more generic “shared interest” interpretation 
often found in the literature. 


What Challenges? 


Votes do not always reflect sincere preferences. Voting occurs in an insti- 
tutional and social setting that shapes incentives for strategic behavior. For 
example, whether decisions are made by simple majority, super majority, 
unanimity, or majority rule with veto players shapes how actors translate 
their preferences into a vote (Maggi and Morelli 2006). Moreover, most 
institutions have strategic agenda-setting processes that determine what is 
and is not brought to a vote. 

Strategic voting raises a number of methodological issues. For example, 
several scholars have pointed out that not all votes are recorded by a roll 
call. The selection process is unlikely to be random (Carrubba et al. 2006; 
Hug 2010). This is probably not a major issue in the UNGA but it is in the 
European Parliament and even more so in the UNSC. Many issues are never 
voted on in the UNSC because states know that they would be vetoed. The 
right to veto is a quintessential example of voting power that could be effec- 
tive even if it is rarely exercised. 

‘The most analyzed type of strategic voting is vote buying: the idea that 
a state can use its economic or military power to influence voting. A well- 
known methodological issue in such studies is endogeneity. For example, 
how do we know that a state who receives a lot of US aid votes more with 
the United States because of the aid or that the United States tends to give 
aid to countries that already had similar foreign policy preferences? There 
are some clever solutions to this issue. For example, nonpermanent UNSC 
members appear to get more favorable IMF loans and more US aid for the 
duration that they hold this seat (Kuziemko and Werker 2006; Vreeland and 
Dreher 2014). The presumed reason is that these countries will support the 
United States should important security issues arise in the Council. Since 
exit from UNSC membership is exogenous (the term is nonrenewable), this 
can plausibly be seen as a causal effect. Since endogeneity issues are not 
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specific to voting analysis, I will not further discuss this issue, other than to 
note that there may be some rare settings where close votes allow scholars 
to use regression discontinuity designs (Voeten 2013). Finally, qualitative or 
anthropological research may help in tracing how influence is exercised in 
the Council (Farrall et al. 2020; Schia 2013). 

There is a second methodological issue: strategic misspecification bias. 
Carter and Stone argue that the credibility of US threats to punish states with 
aid reductions following unfavorable UN votes depends on a state’s regime 
type (Carter and Stone 2015). They estimate an explicit strategic model 
to isolate the degree to which aid influences votes but also to examine the 
degree to which poor democracies share policy preferences with the United 
States. The methodological issue is that omitting this strategic dynamic may 
lead us to overestimate shared preferences among democracies. 

A related strategic issue is bloc voting. For example, the EU coordinates 
the positions of its members on most UNGA votes, making it very difficult 
to disentangle within EU differences in state preferences. The Non-Aligned 
Movement and Group of 77 have long done this for developing countries 
and former colonies, with mixed success. The underlying idea is that these 
states amass bargaining power by voting as a group, which may incentiv- 
ize states to vote differently than they otherwise would. The methodologi- 
cal issue is not unlike that of separating party discipline from ideological 
voting in legislatures: We don’t always know whether Congress people vote 
with their party because they converge ideologically or because the party has 
resources and incentive schemes to keep them in line. IO scholars still have 
a lot to learn from methodological advances in that literature. 


To Go Further 


Armstrong, David A., Ryan Bakker, Royce Carroll, Christopher Hare, Keith T. Poole, 
and Howard Rosenthal, eds. 2021. Analyzing Spatial Models of Choice and Judg- 
ment. 2nd ed. London: Routledge. 

Hix, Simon, and Abdul Noury. 2018. “Power versus ideology: Political group switch- 
ing in the European parliament.” Legislative Studies Quarterly 43 (4): 551-94. 
Voeten, Erik. 2021. Ideology and International Institutions. Princeton: Princeton Uni- 

versity Press. 

Vreeland, James Raymond, and Axel Dreher. 2014. The Political Economy of the United 
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CHAPTER 15 


Statistical Analyses with IO Data 


Fabien Cottier and Heidrun Bohnet 


Over the last two decades, international organizations (IOs) have become 
major data providers. This data is instrumental in supporting scientific 
research and shaping public policies. In this chapter, we discuss how statisti- 
cal analyses of IO data can help answer important research questions, such as 
“what determines refugee flows?” We illustrate our argument by using data 


from the United Nations High Commissioner for Refugees (UNHCR).' 


What? 


By IO data, we refer to data that either has been collected directly by interna- 
tional organizations or has been assembled by IOs from a variety of sources 
(e.g., national statistical offices, specialized agencies) and made available to a 
broader audience. The World Trade Organization (WTO), for instance, col- 
lects much needed data on international trade flows, while the International 
Telecommunication Union (ITU) provides information on the use and 
access to telecommunications across the world. Furthermore, the United 
Nations Population Division produces global demographic projections. 
Reliefweb, a platform setup by the United Nations Office for the Coordina- 
tion of Humanitarian Affairs (OCHA), collects reports from various agen- 
cies on natural disasters. This type of data sharing portal like the one run by 
OCHA or by other actors such as UNHCR have increased communication 


1. The data used is available on UNHCR data platforms: http://popstats.unhcr.org/en/ov 
erview 
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levels and accelerated emergency response (see fig. 8). More broadly, IO 
data, together with statistics, have proved critical for researchers across a 
wide range of disciplines, from economics to political science to agronomy. 
For instance, it has enabled them to study the causes and consequences of 
displacement (e.g., war, human right violations) (Davenport, Moore, and 
Poe 2003; Neumayer 2005; Salehyan and Gleditsch 2006). 


Why? 


Since national bureaucracies rarely collect data beyond their territory, IOs 
have become important actors in collecting and assembling data. Moreover, 
by setting common standards across the globe, IOs play a crucial role in 
ensuring that these can be meaningfully compared across contexts. Social sci- 
ences have, thus, tremendously benefited from IOs efforts in these domains. 
To analyze these data, however, we need the tools of statistics. 

Statistics, or the quantitative analysis of data, generally, allow research- 
ers to build on IO data and help them answer the questions motivating 
their research. As a tool of social inquiry, the goal of statistics is inference. In 
general, inference can be either descriptive or causal (King, Keohane, and 
Verba 1994). Descriptive inference provides estimates of the mean, mode or 
median for a phenomenon of interest. Examples of descriptive statistics are, 
for instance, the median age of Syrians seeking refugee protection in the 
European Union or the average monthly trade flow between the United 
States and China. Causal inference provides estimates of the relationship 
between two variables, such as the intensity of human rights violations and 
the number of refugees (Neumayer 2005; Moore and Shellman 2006), or the 
extent to which displacement contributes to the spread of violence (Salehyan 
and Gleditsch 2006; Bohnet, Cottier, and Hug 2018; Bohnet and Riieg- 
ger 2019). Causal inference makes it thus possible to determine whether a 
hypothesized relationship between two variables exists. 

Statistics contribute to inference in two primary ways: by producing sum- 
maries of quantities of interest and estimating uncertainty. First, by quantity 
of interest, we refer to the parameter(s) governing the distribution of some 
data, such as the median income of refugee households in Lebanon. These 
quantities of interest can either be descriptive (e.g., mean, median, standard 
deviation) or associative (e.g., a regression coefficient). We label our best 
statistical estimates of these quantities summaries. Yet the problem we, as 
researchers, face is that these parameters are typically unknown and must be 
estimated from the data. As a result, these are by nature imprecise. 
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Estimates of our confidence in these summaries, such as the standard 
error of the mean, provide a measure of uncertainty. They inform on the 
range of plausible values of the summaries. The precision of these summaries 
depends on the number of observations. Typically, a few hundred observa- 
tions will yield an estimate of, for instance, the share of people supporting 
policies in favor of refugees, precise to a few percentage points. 

In addition, statistics is a particularly useful tool for causal inference 
as it can account for the possibility that a specific variable does not result 
from a single cause, but that it is the joint outcome of a number of distinct 
processes, which could be correlated with each other. In this regard, linear 
regression is an important technique for researchers to examine the effect of 
human rights abuses on the number of refugees, while controlling for other 
factors known to correlate with refugee flows, such as poverty (Neumayer 
2005). The correlations between these two variables could thus confound the 
estimated effect of human right violation on refugee flight. 


How? 


As a first step, we encourage researchers to review the documentation to 
understand the definitions employed by IOs, the format of the data and 
possible limitations. For a concrete example, UNHCR provides informa- 
tion on refugee stocks in a dyadic format (pairs of countries of origin— 
asylum). This has two implications. First, for applications (as our own) 
in which researchers study the determinants of refugee flows, it is useful 
to aggregate the data by country of origin, resulting thus in a monadic 
format. Second, while the data measures refugee stocks in a given period, 
we are interested in refugee flows. Hence we generate a variable measuring 
the number of new refugees by subtracting the number of refugees in the 
previous year from the total for the next year. Negative refugee counts are 
censored at zero.” 

In a second step, researchers should examine their data with the help 
of summary statistics (e.g., mean, median, standard deviation). In doing 
so, it is good practice to question whether the information reported agrees 
with one’s own prior knowledge. Discrepancies could alert the researcher 
to problems with the data, yet they could just as well indicate that existing 


2. Formally, N new refugees = N refugees — N refugees in the previous year if > 0, oth- 
erwise 0. 

3. High rates of unreported country of origin for UNHCR refugee data is also a major 
concern. For a discussion and approaches to overcome this problem, see Marbach 2018. 
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knowledge may be incomplete. In general, we encourage researchers not to 
limit themselves to considering only summary statistics, as significant pat- 
terns in the data may be hidden. Hence it may be useful to explore how the 
data varies temporally and spatially. 

In a third step, we use the available data from UNHCR to determine 
to what extent authoritarian regimes, human rights violations, and armed 
conflicts cause displacement using /inear regression. As the name indicates, 
we assume a (/og) linear relationship between these variables and displace- 
ment. Data for these variables comes from the vdem dataset (Coppedge et al. 
2019),* the Political Terror Scale (Gibney et al. 2019),’ and the UCDP Armed 
Conflict Dataset (v 19.1) (Gleditsch et al. 2002; Pettersson, Hégbladh, and 
Oberg 2019). In addition to low and high intensity, we distinguish between 
interstate and internal conflicts.° Finally, we control for GDP per capita and 
population (World Bank 2019) and add country and year fixed effects.’ 

Table 6 presents the results of the regression. The first estimate for each 
variable is the coefficient, which can be interpreted as our “best guess” for the 
direction and magnitude of the relation between a predictor and the depen- 
dent variable. The number in parentheses below is the standard error, which 
is an estimate of uncertainty. The larger the standard error relative to the 
coefficient, the more uncertain the association. Asterisks indicate whether 
the T value (the ratio between the coefficient and the standard error) has 
crossed a p-value threshold.* 

Table 6 is an analysis of the determinants of refugee flows over the period 
1990-2018. The results indicate that armed conflicts are a primary cause of 
refugee flows. Both civil and international wars are associated with a sub- 
stantial increase in the number of refugees, the more so for high intensity 
conflicts. Similarly, human rights violations have a large effect on flight. A 
decrease of one point on the political terror scale (i.e., more severe violations) 
more than doubles the number of refugees. While the coefficients for demo- 
cratic regimes are suggestive of a negative association between democracy 


4, The index (polyarchy) measures electoral democracy on a scale from 0 (authoritarian) to 
1 (full democracy). We add a squared term to model a possibly nonlinear relation. 

5. Formally, it measures human rights violations by agents of the state on a five-point scale. 

6. High-intensity conflicts cause at least a one thousand battle deaths in any given year. 
Conversely, low-intensity conflicts are periods during which violence causes between twenty- 
five and one thousand battle deaths. 

7. The fixed-effects control for unobservable factors, which may have affected displace- 
ment patterns and are correlated with the independent variables. 

8. While useful in helping to read statistical tables, we would advise researchers to cau- 
tiously approach p-values. Their use has been the target of criticisms by statisticians (American 
Statistical Association 2016). 
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Table 6. Statistical Analyses Using UNHCR Data 


Polyarchy -4.315 
(3.363) 
Polyarchy2 3.200 
(3.519) 
Human rights violations -0.804*** 
(0.125) 
International war (low) 0.869** 
(0.342) 
International war (high) 6.496*** 
(1.574) 
Civil war (low) 0.652** 
(0.251) 
Civil war (high) 2.420" 
(0.490) 
GDPpc log -1.660*** 
(0.327) 
Pop log 2.115" 
(0.658) 
Country, Year FE Yes 
Observations 4,259 
Adjusted R2 0.309 


Note: *p < 0.1; **p < 0.05; ***p < 0.01 


Std errors clustered by countries in parentheses. 


and refugees, this association is not statistically significant, thus precluding 
any conclusion. Finally, the results indicate that economic development is 
negatively associated with refugee flows, while the demographically larger 
countries, generally, generate more refugees. 


What Challenges? 


Conducting research using IO data raises specific challenges. We discuss 
three aspects: definitions, measurement, and the unit of analysis. Research- 
ers should be alert to these concerns and carefully assess the validity and 
reliability of their data.? While we illustrate our discussion with UNHCR 
data on refugee flows, readers should be mindful that concerns pertaining to 


9. Statistics hinge crucially on key assumptions. It is beyond the scope of this chapter to 
discuss these. Interested readers should consult introductory level statistical literature, such 
as Agresti (2017). 
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data produced by IOs are frequently idiosyncratic to one’s own specific line 
of inquiry. Readers are therefore invited to carefully appraise IO data in their 
own field of investigation. 

First, the definitions employed may make comparisons difficult. In our 
illustration, different IOs (e.g., UNHCR, African Union) employ slightly 
different definitions of “refugee.” The fluidity of refugee, internally displaced 
person (IDP), or returnee categories is another difficulty. For example, peo- 
ple from South Sudan often move back and forth between Ethiopia, South 
Sudan, and Uganda. Within the same year, one person can successively 
belong to each of these three categories. Another challenge is that UNHCR 
data only reflects the situation at the end of the year. Therefore if refugees 
flee a country and return within the same year, these movements won't be 
recorded. 

A second challenge regarding data provided by IOs concerns measure- 
ment bias. For example, researchers should be aware of risks of politiciza- 
tion of the data. In fact, this challenge is magnified as IOs depend largely 
on data collected by national bureaucracies, in particular for demographic, 
health, as well as economic indicators. With regards to refugee specifically, 
both hosting and sending states, for instance, have incentives to downplay, 
or conversely exaggerate, the number of refugees originating from a par- 
ticular country, because refugees are “a [potent] symbol of state failure” 
(Crisp 1999: 8). Conversely, refugees originating from allied countries may 
be labelled “economic migrants.” Refugee counts might also be inflated in 
order to attract more international aid. For instance, Uganda refugee num- 
bers were revealed to have been exaggerated by 300,000 (Okiror 2018). In 
addition, some people do not register as refugees in the hope of evading 
detection. Hence displacement figures may not necessarily reflect the true 
picture. 

Finally, researchers should be aware of the appropriate unit of analysis 
when analyzing data from IOs. For example, Salehyan and Gleditsch (2006) 
found out that the more refugees a country received, the more likely it was 
to experience civil war. Yet in a new study, Zhou and Shaver (2021) have 
challenged this conclusion using data on refugee settlement locations. At the 
same time, subnational data on refugees, such as data on settlements, is often 
harder to obtain and come with important limitations, such as approximate 
location and missing refugee figures (see Bohnet 2015). 
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BOX T 


Navigating Human Resource Statistics 


Fanny Badache 


Researchers are increasingly interested in examining the composition of 
United Nations (UN) personnel regarding sociodemographic characteris- 
tics. In particular, existing studies focused on the composition of the staff in 
terms of nationalities (Parizek 2017; Novosad and Wercker 2019; Badache 
2020; Oksamytna, Bove, and Lundgren 2021) and gender (Haack 2014). 
However, human resource data such as nationality, gender, and age of the 
personnel of UN organizations are not easily accessible and, when they are, 
can be partial. Available data can be used for certain purposes but also have 
some caveats. 

Two sets of human resource data concerning UN organizations are pub- 
licly available. First, the UN Secretariat is the only UN organization that 
overtly publishes data on the composition of its personnel. These reports 
have been published on a yearly basis by the secretary-general since 1948. 
Initially limited to staff in posts subject to geographical distribution, their 
breadth has increased over time. Since 2007, these reports provide data on 
the nationality, age, contract type, and gender for all its staff irrespective of 
the categories of service and funding sources. The main advantage of these 
reports is to present raw information by countries. The comprehensive sta- 
tistical tables at the end of these reports provide, for each member country, 
the number of staff by categories of service, appointment types, and gender. 

‘The second available source includes the statistics gathered by the Chief 
Executives Board for Coordination.’ They provide human resource data for 
all UN organizations on age, staff location, nationality, gender, length of 


1. https://www.unsystem.org/content/un-system-human-resources-statistics 
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service, grade, and category of service. These statistics are graphically pre- 
sented on the website and can be downloaded. This database is very useful 
to make comparisons between UN organizations. However, since they are 
presented by organizations and by characteristics, they cannot be used for 
cross-sectional analyses such as studies that address the composition in terms 
of age and gender. Another limitation of this set of HR data is that they are 
limited to staff with fixed-term contracts of one year of more. 

In addition to the above-mentioned limitations, it must be noted that 
the available data on UN personnel is limited to a small number of sociode- 
mographic attributes (i.e., age, gender, nationality). In order to collect more 
human resource data such as education, professional background, and social 
origins, researchers should use other data collection tools such as conducting 
biographic interviews (see chapter 6—Biographic Interviews) and create their 
own database (see box u—Building Databases on Individuals). 
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CHAPTER 16 


Large-N Data and Quantitative Analyses 


Charles B. Roger 


Prior to the 1990s, large-N data and quantitative studies of international 
organizations (IOs) were relatively rare. Case studies were the dominant 
mode of analysis. Since then, however, there has been a significant shift. 
Large-N data and quantitative methods are now well-established. Scholars 
have also increasingly combined quantitative and qualitative approaches to 
provide more compelling answers to empirical questions than either could 
in isolation. This has had profound effects. Quantitative and mixed-methods 
analyses have significantly advanced our understanding of IOs and gener- 
ated new puzzles to explore. These approaches are, therefore, important for 
scholars to understand and appreciate and should be an essential part of 
the toolkit for understanding IOs. However, quantitative methods have also 
presented unique challenges when applied to IOs, which researchers should 
be aware of. My aim in this chapter is to provide some insights into the kinds 


of things that should be kept in mind. 


What? 


“Large-N data,” here, refers to datasets that quantify IOs, specific features of 
IOs, or IO activities. These come in several forms. Time-series datasets con- 
tain observations about a single unit across time—for instance, how member- 
ship in a single IO has changed. Cross-sectional datasets contain observations 
across many units, often at a single point in time—i.e., variation in member- 
ship across all “currently active” IOs. Panel data combine information of both 
types—i.e., how membership across IOs has changed over time. In practice, 
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a range of datasets exist providing different information about IOs, and these 
can be found either in standalone files online or in the replication files for par- 
ticular papers. A few of these are discussed later. A scholar may also, of course, 
collect original data through observations or experiments. 

However structured or attained, large-N datasets are widely used for 
important descriptive purposes. Liesbet Hooghe et al. (2019) have, for 
instance, collected data on the authority of IOs, showing how this has varied 
throughout the postwar period. But such datasets are also frequently used 
to make causal inferences by employing them in “quantitative analyses.” In 
the case of Hooghe et al., they use their data to explain the growth of IO 
authority. Different statistical models will typically be used to do this, which 
depend on the kind of dataset one is working with. Specific models have 
been developed to handle time-series data or panel data, variables that are 
continuous or limited in different ways, and so forth. Scholars often develop 
expertise in working with particular models and make use of different soft- 
ware packages to execute them. They also frequently employ tests to verify 
whether statistical assumptions hold and will undertake sensitivity analyses 
to determine whether findings change when variables are measured differ- 
ently or when employing different modeling strategies. 


Why? 


Quantitative methods offer two distinct benefits. First, they can help us 
understand complex patterns and processes that would otherwise be impos- 
sible to observe. Compared to case studies, quantitative data can be used to 
summarize large-scale phenomena, like membership patterns, voting within 
IOs, or types of IO behavior. This kind of information moves beyond single 
observations and allows scholars to identify “aggregate” patterns, which we 
might suspect are true but would not be able to say so confidently otherwise. 
Work by Tallberg (2013) on “transnational access” to [Os—participation 
by nonstate actors in IO decision-making—provides a wonderful example. 
Earlier researchers observed that access had increased in several prominent 
institutions but it was unclear whether this pattern was new or widespread 
(O’Brien et al. 2000). By collecting data across a large sample of IOs, 
Tallberg et al. could answer this question effectively, showing that it had 
increased, on average, since the 1950s. Without this, it would have been 
impossible say for sure. 

The second advantage is that quantitative methods can deepen our 
understanding of the relationships between variables. Many of the processes 
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scholars analyze are complex. In a single case, or even several, a change in 
a variable may not lead to a change in another due to a range of idiosyn- 
cratic or confounding factors. Equally, even when an association is observed, 
this need not mean they are causally linked. It may simply be coincidence. 
To evaluate arguments convincingly, therefore, we often need to determine 
whether associations are systematic or due to random chance. This is where 
statistical methods and large-N data come in especially handy. By look- 
ing across many cases and leveraging the logic of probability, we can use 
statistical models to separate the “signal” from the “noise” and reach more 
convincing conclusions about underlying relationships. Work by Tallberg 
again offers a compelling illustration. Earlier studies suspected that protest 
frequently led IOs to “open up.” However, their analysis revealed that this 
effect was not so pronounced. ‘They found, instead, that other factors, like 
functional demand and the democratic values of member states, were more 
powerful causal drivers. It would have been challenging to reach this conclu- 
sion without reliance on quantitative methods. 


How? 


‘The basic starting point for any quantitative analysis of IOs is the selection 
of a large-N dataset. This initial choice, and many decisions that follow, will 
largely be determined by theory. Depending on the question one asks about 
IOs, different datasets will be relevant. In my recent research, for instance, 
I have been interested in the legal nature of IOs and to study this I used 
a dataset that measures levels of legal formality (Roger 2020). This is the 
“dependent variable” in my analysis. However, in practice, a large number 
of options are available, and data on IOs can be used as either dependent 
or independent variables (Gartzke and Schneider 2013). One of the earliest 
and most frequently used IO datasets was developed as part of the Cor- 
relates of War (COW) project initiated by J. David Singer (Wallace and 
Singer 1970; Pevehouse, Nordstrom, and Warnke 2004). The most recent 
version of this dataset measures membership in formal bodies and has served 
as a fundamental resource for IO scholars. Today, however, we also have 
fairly comprehensive data on features of 1Os—their independence, policy 
scope, voting rules, and so on—and many of their activities too (Haftel and 
Thompson 2006; Hooghe, Lenz, and Marks 2019). 

Once one has selected data that measure the key variable of interest, 
the task is then to combine this with data on other covariates—democracy, 
economic openness, national capabilities, and so on. The data that are rel- 


226 International Organizations and Research Methods 


evant will again be determined by theory. In my case, I needed data that 
theory suggested might shape the legal design of IOs. To test “power-based” 
explanations, for instance, it seemed reasonable to measure power using pre- 
existing data on GDP across relevant states. A similar process was used to 
choose other variables and controls, and in most cases the existing litera- 
ture provided a helpful guide to what would be appropriate. Once data are 
assembled, the next task is to undertake an analysis. The specific type of 
analysis that will be appropriate will largely depend, as mentioned, on con- 
siderations related to the outcome of interest. In my study of informality, 
the data were cross-sectional in nature and the key variable was binary. This 
meant that certain types of limited dependent variable regression models 
would be needed (for a discussion, see Long 1997). Others may use entirely 
different approaches. But in each instance, the end goal will be the same: the 
analysis should help to discern which relationships may be due to more than 
chance and how substantively important these are for the final outcome we 
care about. 

Things are not finished there, however. Typically, as noted above, a 
scholar will try to test the robustness of the findings by using different mod- 
els, adding and dropping variables, measuring key ideas in different ways, 
and so on. The idea behind all these moves is to check whether results are 
sensitive to minor changes. Ultimately, if they fail these tests, our confidence 
in a finding will be correspondingly reduced; but if not, we may remain 
confident that the relationships we have uncovered are systematic in nature. 


What Challenges? 


When performed well, quantitative analyses can be a powerful tool. But they 
can have disadvantages. Many of these are general in nature. Quantitative 
methods are often less useful, for instance, for understanding the precise 
nature of the mechanisms linking variables. We may uncover associations 
suggesting that some link exists, but there may be several plausible accounts 
of the mechanism at work. When this is so, statistical approaches reach their 
limits. It will often be valuable to combine quantitative methods with quali- 
tative ones. By leveraging the unique inferential capabilities of both, it is 
possible to compensate for the weaknesses inherent in each. A growing lit- 
erature explains how this can be done, and such mixed-method studies are 
now widely used in the study of IOs. 

In addition to such drawbacks, however, there are others more specific to 
studying IOs. The first is that IO data can sometimes be uneven and difficult 
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to work with. Much depends on the question one asks, but for research that 
focuses on the “IO” as the unit of analysis, or on particular processes within 
IOs, it can be difficult to attain the data needed. A key issue arises from the 
fact that most datasets include samples of IOs or, when more complete, may 
be structured for cross-country or country-year analysis, i.e., where the unit 
of analysis in such studies is a state-level “event” in a given year. State-IO 
membership data, such as that contained in the COW dataset, is structured 
in this way and can be integrated fairly easily with other types. But if one is 
interested in understanding what determines the design of IOs, it will often 
be necessary to match different datasets and (sometimes) transform existing 
data to ensure it is structured appropriately. Because many include different 
samples of organizations and do not always match up perfectly, this can 
be challenging. The need to transform data can also mean that research on 
IOs—of this type—can still be quite costly and time-intensive. 

Second, even when available, it is important to approach datasets with 
some caution. There are, for instance, known problems in the COW dataset: 
duplicates, incorrect dates, missing IOs, and so on. A more fundamental 
issue arises when the concept underlying the data is problematic, or doesn’t 
align with that presumed by a theory. One example of this comes from data 
on voting rules. Blake and Payton (2015) offer comprehensive data on vot- 
ing arrangements in IOs. But others have questioned how they define these, 
especially the practice of “consensus” voting. They code consensus as being 
synonymous with “unanimity,” yet Erica Gould (2017) has argued that 
this is often not the correct interpretation, and that when we take this into 
account the causal picture looks different. In my work with Sam Rowan, I 
have shown that much research focuses on a narrow range of “formal orga- 
nizations” (Roger and Rowan 2022). But there are, in fact, a number of 
“informal” bodies left out of such analyses. If these are not included, this 
can sometimes affect findings. So it is important to be conscious of these 
issues when using data “off the shelf” and to seek a close alignment between 
measures and key concepts. 

To do this, it is sometimes necessary to collect your own data. Generat- 
ing your own data can ensure a good match between the data and ideas 
a researcher is working with. It can also reveal new puzzles. In practice, 
the Yearbook of International Organizations has proved to be an invaluable 
resource for doing this, since it contains descriptions of numerous institu- 
tions (Union of International Associations 2009). It has correspondingly 
served as the starting point for many datasets, COW included. Other 
resources include the websites, legal documents, and publications of IOs. 
These can often be useful but may not be available for historical institutions, 
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and can be somewhat uneven in terms of their content. It is also worth 
noting that they can sometimes give a misleading picture of what happens. 
A charter may outline a precise set of decision-making procedures and a 
website may offer helpful descriptions, but practices may differ. To handle 
this, surveys and survey experiments have increasingly been used to collect 
data (Hooghe 2005; Hardt 2018). These approaches, described elsewhere 
in this volume, can help to measure otherwise hidden aspects of IO design 
or behavior and the datasets generated can contribute powerfully to our 
understanding of IOs. 


To Go Further 
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CHAPTER 17 


Computerized Text Analysis 


Mor Mitrani and Inbar Noy 


This chapter presents the potential uses of computerized textual analysis 
methods in studying international organizations (IOs). It shows how these 
methods are useful for studying both the organizations themselves and 
the interactions among their members. These innovative methods allow 
researchers to analyze extensive data, identify latent discursive aspects of 
texts, and extract a tone or a position regarding a certain issue. 


What? 


Text analysis is a term for a large array of methods that use textual data to 
scrutinize social reality. In political studies, the premise that language, text, 
and communication are pivotal elements in political processes yielded over 
the past decade political research that treats text as data and applies human 
and computerized methods for its analysis. Technological advancements that 
facilitate extensive digitization of political texts and allow systematic mining 
and processing of large-scale corpora have increased the popularity of com- 
puterized methods (Grimmer and Stewart 2013; Lucas et al. 2015; Monroe 
and Schrodt 2008). Their application is mostly oriented toward identifying 
latent political processes through the analysis of text and language. Most of 
these studies have invested extensive effort in developing models that use 
“text as data” in order to measure, score, and scale the political positions 
of political actors such as parties, legislators, and interest groups based on 
automated text analysis (Laver, Benoit, and Garry 2003; Slapin and Proksch 
2008). 
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‘These approaches, however, have rarely been exported to the international 
arena. Studies that applied computerized text analysis in IR usually either 
utilized a lexical-based approach to coding event data based on newswire 
reports (King and Lowe 2003) or performed automated coding of NGO 
reports, mainly in the field of human rights (Fariss et al. 2015). A few studies 
treated international texts produced in international settings—notably by 
IOs—as data that can provide insights into the processes of agenda-setting, 
political attention, and political positions (Alschner and Skougarevskiy 
2016; Bagozzi 2015; Gurciullo and Mikhaylov 2017). Some studies have 
aimed at mapping positions of IO members regarding a specific issue or 
policy (Bagozzi 2015; Frid-Nielsen 2018) or to examine the influence of 
interest groups on choices of policy outcomes (Kliiver 2009). Other studies 
have also applied methods of computerized textual analysis for the purposes 
of analyzing the content of a certain agenda, and the changes of framing of 
an agenda over time (Park, Murdie, and Davis 2019; Schönfeld et al. 2018). 


Why? 


‘The international political sphere is rich in texts, is built out of texts, and to 
a great extent relies on and is realized by discursive and textual interactions. 
Public discourse at the international level can serve as a source of data, and 
existing computerized methods—for big data—can systematically examine 
both the content of and discourse on the interactions that eventually design 
our main subject matter—world politics. These interactions often take place 
in the institutional contexts of international organizations at two levels. First 
is at the micro level, through the day-to-day natural language of diplomatic 
and political communication in bilateral or multilateral settings. Second is 
at the macro level, through texts that institutionalize international relations 
through legal language, including treaties, charters, agreements, and other 
reports and policy documents, that shape international norms and practices. 
IOs are thus great sources of textual data that can be used to study the 
organizations themselves, their member states, and the political relations 
that are constructed in these contexts. This is more than a methodological 
claim. Indeed, IOs produce dense and extensive amounts of text, and there- 
fore it is methodologically useful to apply computerized text analysis when 
studying them. However, we should not treat text as just a methodologi- 
cal resource for data but rather analytically as a fundamental political tool 
through which actors present identities and construct (political) relations 
through the mechanisms of legitimacy and identification. 


232 International Organizations and Research Methods 


Textual analysis allows collecting and processing large corpora at the 
organizational level, which can generate broad longitudinal data on all IOs 
and from varying perspectives, with a scope that human coding or analysis 
is incapable of. Recently, IO scholars have focused on establishing a rule- 
based perspective on IOs that focuses on connection between legitimacy 
and deliberation. The main premise here is that IOs and states (in their con- 
text) continually negotiate a set of rules and norms that shape their mutual 
interactions, roles, and relations and establish standards of legitimacy (Hurd 
2007; Johnstone 2011). By analyzing the tone of texts, as well as their fram- 
ing, textual analysis is a valuable tool for understanding the range of activi- 
ties in which an IO can legitimately engage and identify norms and rules 
of behavior. Text analysis can also assist in identifying rules regarding the 
expected behavior of members of a certain IO. 

Text analysis can also shed light on both the content of IOs’ agendas 
and of the processes of agenda-setting by illuminating aspects of framing, 
positioning, and priming. It can provide valuable tools for comparison of 
similarities and differences in issues and agendas between organizations or 
among their members. Such analysis can also discern issue-linkage and diffu- 
sion between organizations or between an organization’s members. Comput- 
erized textual analysis also enables researchers to assess the discursive power 
of political actors within organizations, and how they construct and reframe 
certain issues, and their ability to shape organizational agendas. Finally, ana- 
lyzing the content of IOs’ documents, as well as speeches of IO officials, may 
provide an understanding of the norms and legitimization aspects within an 
organization. 


How? 


Prior to applying computerized textual analysis, a researcher needs to decide 
which texts and databases will be analyzed. In the process of text selection, 
a researcher can either choose to analyze textual databases that already exist 
or create new databases using tools that enables importing and collecting 
of data. In most cases, the researcher will be required to validate the texts 
and prepare them before processing. Preprocessing requires the researcher 
to first discard words and symbols irrelevant to the analysis—such as punc- 
tuation, commonly used words (“stop words”), or numbers. Additionally, 
the researcher may want to remove words that occur at a low frequency 
in the texts, as they may be irrelevant to the analysis. Finally, the research 
may recognize different forms of a certain word by stemming or lemmatiza- 
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tion (Grimmer and Stewart 2013). The next section will elaborate on three 
methods of computerized textual analysis—topic modeling, Wordfish, and 
sentiment analysis—and their application in IO research. 


Topic Modeling 


Topic modeling is a collection of unsupervised methods that compile sets of 
words and attribute each set to a different topic. The algorithms identify the 
topics by extracting patterns of word recurrence, to the extent that they can 
be acknowledged as assembling a certain topic (Grimmer 2010). For exam- 
ple, as words that are used as greetings, such as “thank you” and “congratula- 
tions,” usually occur together, they will be recognized as belonging to a cer- 
tain topic (Watanabe and Zhou 2020). In social sciences, topic modeling is 
used mainly through the latent Dirichlet allocation model, which examines 
the distribution of topics within a certain document by identifying repeated 
patterns of cooccurrences of words and/or appearances of the words outside 
the topics to which they were attributed (Blei, Ng, and Jordan 2003; Chang 
et al. 2009; Watanabe and Zhou 2020). The application of topic modeling 
in political science has focused on examining the content of political actors’ 
agendas (i.e., Grimmer 2010). In IO research, topic modeling was thus far 
applied mainly to examine changes in framing of a certain issue over time 
among different IOs (Park et al. 2019) as well as to assess the impact of posi- 
tions of members of an IO regarding a certain issue (Bagozzi 2015). As topic 
modeling can be applied to detect latent agendas of certain actors (Grimmer 
2010), its application in IO research can provide a comprehensive under- 
standing of agenda salience within an IO framework. 


Wordfish 


Wordfish is a model that aims to identify actors’ positions on a certain issue 
based on one-dimensional scaling of word frequencies in texts (Slapin and 
Proksch 2008). Wordfish scaling is unsupervised and unfixed in advance. 
Consequently, it requires the researcher to retrospectively assess the results 
in order to define the scaling dimensions. As Wordfish enables comparative 
examination of actors’ positions, it is mostly applied in political science in 
comparative studies examining parties’ positions on specific topics, or in 
temporal analyses of changes in actors’ positions (Lauderdale and Herzog 
2016; Slapin and Proksch 2008). The Wordfish tool can be extremely useful 
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in the study of IOs for both inferential and comparative purposes. It can 
deductively expose the positions of certain organizations—or of organiza- 
tion members—regarding a specific issue (Slapin and Proksch 2008: 711). 
For example, Frid-Nielsen used Wordfish in order to estimate the positions 
of members of the European Parliament (MEP) regarding asylum policy 
(2018). Based on relative word usage of MEP’s speeches on this contested 
issue, Wordfish provides an initial scale of the range of positions and enables 
the researcher to compare the positions of the MEP examined. 


Sentiment Analysis 


Sentiment analysis is a supervised dictionary-based automated analysis that 
aims to detect and analyze different aspects of sentiments in political texts 
by assessing the tone of the text as negative or positive (Proksch et al. 2019; 
Young and Soroka 2012). Thus it can assist in understanding attitudes and 
opinions regarding a certain topic. Based on a chosen dictionary that sets a 
list of negative and positive words, the automated sentiment analysis mea- 
sures the frequency of the relevant words in the unit of analysis. In political 
science, this approach is mostly applied in political communication and is 
used to assess the way that political issues are represented and framed in the 
media and by users of social media (Soroka, Young, and Balmas 2015). 

Together with other tools, sentiment analysis can provide an understand- 
ing of the normative positions of organizations and their members on certain 
issues and thus can insinuate which agendas are most likely to be promoted 
and simultaneously which issues are considered delegitimate or wrongful 
from the organizational perspective. Sentimental dynamics across time and 
among states or organizations can illuminate processes of politization and 
identify who shapes them and which tools they use. 


What Challenges? 


In the attempt to apply computerized text analysis to the study of IOs, three 
main challenges must be acknowledged. ‘The first challenge is the issue of 
translation. In an attempt to study a great variety of actors and organiza- 
tions, we often rely on translated texts that are provided by the organizations 
themselves. Translation is often insensitive to cultural differences, involves 
personal decisions, and is also highly prejudiced toward specific institutional 
and organizational discourses. The second challenge in applying computer- 
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ized text analysis is the tendency to overlook the context. Automated analy- 
sis of very large amounts of text with no interpretative elements may reduce 
the reading of the texts to the level of word frequencies and cooccurrences 
and therefore ignore deeper or complex discursive elements of context, lexi- 
cons, and framing. The third and last challenge is the challenge of genuine- 
ness and intention, or more precisely, the need to deal with the question: 
“what if people do not mean what they are saying.” 

All three challenges raise valid and important questions at the theo- 
retical and methodological levels. They can all be accommodated once the 
researcher adopts three main principles in their research design. First, text 
analysis will always be limited to what texts can tell us. In this case, texts are 
not reflections of thoughts or mirrors of reality. Text is a social tool through 
which people choose how to present their reality, tell their stories, and 
interact with fellow agents, and it is thus a valuable tool for exposing how 
these political processes take place. Text analysis is therefore more effective 
in answering “how” questions than “why” questions. Second, computerized 
text analysis is a great tool for distilling findings from a large amount of data, 
but it can always benefit from complementary human analysis of specific 
cases. Third, one must learn to work with what there is and be mindful 
of the limitations. International political discourse does not take place in 
organized or ordered settings. On the contrary, it is multilingual; it happens 
(simultaneously) in many institutional and organizational arenas (see chap- 
ter 11—Discourse Analysis). It has no ground rules or effective authority to 
govern or discipline it, and in these respects, it fully represents the anarchic 
nature of the international arena. This is not a good reason to disregard the 
extensive amount of texts and discursive interactions that construct it, just 
the opposite. Words are the most accessible and valuable data source for 
researchers of international politics in general and of IOs in particular. If we 
learn to acknowledge that this data source is not (and cannot be) sterile or 
perfect but still allows us to build valid datasets and design reliable research, 
we can certainly expose a fundamental and hitherto quite under-researched 
layer of international politics. 


To Go Further 
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CHAPTER 18 


Multiple Correspondence Analysis 


Constantin Brissaud 


How should we classify international organizations (IOs)? Or central bank- 
ers in the global field of power? Though rarely used in international rela- 
tions, Multiple Correspondence Analysis (MCA) is an inductive mode of 
classification that is especially useful for this kind of task. It allows the study 
of actors positioned in a given field—such as the field of international exper- 
tise or international organizations—and to verify whether their position in 
the field produces effects on their positions-taking. 


What? 


To study a population, it is often useful to determine its principles of cohe- 
sion. MCA allows researchers to inductively create subgroups of individu- 
als characterized by common properties (or modalities of variables). It is 
a statistical method that synthesizes a set of categorical variables in one or 
more two-dimensional spaces (e.g., paper sheets). It summarizes the infor- 
mation contained in a dataset, by showing how certain modalities of vari- 
ables are frequently found together within statistical individuals. Symmetri- 
cally, the method yields cohesive—in that they share common modalities of 
variables—subgroups in the statistical population, be it composed of inter- 
national civil servants, experts, international projects, or even international 
organizations. 

Since MCA yields cohesive subgroups that have statistically significant 
modalities of variables in common, it allows the researcher to determine 
whether these common properties determine a common behavior. Finally, 


238 


Multiple Correspondence Analysis 239 


MCA is a relational method: the position of a statistical individual on the 
plane is to be understood in comparison to the positions of others. It is 
therefore the best method to objectify a field in the Bourdieusian sense. 

MCA has initially been used in sociology by Pierre Bourdieu and his 
colleagues to map the “field of power.” For researchers who study the field 
of power, the treatment of prosopographical data (Lebaron 2009; see also 
chapter 26—Prosopography) through MCA can show the link between a 
given set of modalities of variables (for example educational background, 
inherited socioeconomic status, and so on) and a certain type of behavior. 
The method therefore shows the oppositions characterizing a social space 
through the comparison of groups of individuals (or sets of properties) on 
the axes. It indeed yields new variables, called factors or axes, that summarize 
information from a large number of variables. 

MCA creates clusters of statistical individuals that share a common set 
of properties and groups them on a geometric plane. Two planes are then 
created: the plane of individuals—where one can see clusters of individuals, 
the clusters being the product of individuals sharing common properties— 
and the plane of properties. If the properties are close on the plane, it means 
that individuals retaining one of those properties are very likely to retain the 
others. Symmetrically, two individuals that are close on the plane will share 
a lot of common properties. Since properties are held by individuals, the two 
planes are two representations of the same social space. 


Why? 


At the organizational level, the modes of financing, the variety of countries 
involved, the capacity to carry out external actions, or to raise funds are 
often perceived as structuring differences in the world of international orga- 
nizations. If studying international spaces at the individual level, properties 
such as nationality, education, or trajectory differences are routinely under- 
stood as having deep impacts on agents’ behavior, and on decisions-taking 
processes (Brissaud and Juven 2020), but few studies have relied on MCA to 
explore the field of IOs. MCA allows researchers to inductively create groups 
of actors or groups of IOs to describe the (statistical) population and verify 
that these groups are coherent, in the sense that they behave the same way. 
Vincent Pouliot has used this method to map the pecking order of coun- 
tries in two international peacekeeping organizations, NATO and the UN 
(2016). Thanks to MCA classification, he showed the variations of coun- 
tries’ standing echelons and classified country behavior in both IOs. Médéric 
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Martin-Mazé (2018) used MCA as well to unravel the complex relations 
between technical assistance agencies. He positioned the various interna- 
tional organizations dedicated to border monitoring in a transnational space 
constructed by the projects those organizations conduct. Both Pouliot and 
Martin-Mazé, therefore, used MCA to show the main principles of differen- 
tiation within a complex population of organizations. 

However the clusters created through MCA are not only descriptive: 
since they show unexpected similarities between individuals close to each 
other on the graph, and oppositions between the most distant of them, they 
are also very strong incentives to stimulate sociological imagination. My 
own research specifically aims at understanding how the OECD has become 
a central IO in the field of international health reforms. Deriving from the 
sociology of intellectuals (Lamont 1987), one explanation is that to gain 
legitimacy in a given field, an intellectual (or in this case an organization) 
must be associated with well-known authorities in the field. MCA on the 
authors’ population of OECD reports on health tends to show that the first 
reports were signed by well-known economists, while recent OECD reports 
on health are signed by “data-crunchers” whose volume of academic capi- 
tal is much less important. It then tells a story on the way OECD gained 
legitimacy in the international field of health care: by enrolling well-known 
academics, OECD legitimated its data and its recommendations, therefore 
being more and more used by governments, other international organiza- 
tions, and academics, as in a virtuous circle. 

Used in multimethod research design, MCA also allows researchers to 
relate the behavior of observed or interviewed individuals to their position 
in the space statistically constructed. And finally, as MCA unveils the most 
structuring properties of the space, it also allows researchers to compare vari- 
ous social spaces in order to determine whether they have the same structur- 
ing properties, and therefore build bridges between national and interna- 
tional levels of analysis (Gayon 2020). 


How? 


To conduct MCA, scholars first need to create a database of the statistical 
individuals or organizations, with statistical units in rows and properties in 
columns. Information about properties is then gathered through the Internet 
(mainly LinkedIn and academic CV) and interviews (when database consists 
of individuals). Scholars distribute these pieces of information in groups of 
variables (in columns) that provide a point of view on the population under 
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study, built according to existing literature, ethnography, and interviews. 
The modalities of variables must be qualitative and concern about the same 
number of individuals each time. The very description of individuals there- 
fore stems from research choices: a statistical individual may be described by 
a potentially infinite number of variables. The variables finally retained are 
those the researcher assumes to produce effects on individuals’ behaviors. 
Variables that are taken into account and their modalities should be justi- 
fied and relevant regarding the population studied. The decision to keep or 
exclude a variable, or to categorize it in a certain way, is a deeply scientific 
question—as opposed to a technical question—that can totally change the 
results. On the graph, each group of variables is finally represented with a 
different type of point. 

In my work on the OECD, I crafted a database of the 109 experts who 
signed the OECD reports on health by coding some of their properties with 
qualitative variables. Information gathered on each expert is distributed 
into five groups of variables (in columns) distributed into nineteen active 
variables counting fifty-four modalities and twenty-four additional variables 
(not taken into account in the MCA) with a total of seventy-two modali- 
ties. For example, since internationalized trajectories or the ability to speak 
several languages had been outlined as a powerful asset to work in IOs, I 
decided, in my research, to construct a group of variables on the “cosmo- 
politan capital” retained by each individual. Each variable included in the 
analysis is indeed a hypothesis on the structure of the population. 

Once the dataset is constructed, MCA can be run thanks to different 
platforms such as the package FactoMineR (Lé et al. 2008) from the sta- 
tistical software R.' Applied to the database, MCA produces a table that 
gives, for each modality of variable, its contribution to the production of 
axes.” It also produces another table that gives the positions of modalities 
of variables and the positions of individuals on the graph. Then it plots a 
two-dimensional graph that must be improved using other packages such as 
geplot2, showing the modalities of variables on a plane. The researcher must 
then look at the percentages of variance explained by every axis. It is a mea- 
sure of the proportion of the differences between individuals that is summa- 
rized by the axis. Then the researcher chooses the number of axes that must 
be retained in the analysis (often two or three).’ The plots should be read 


1. Many tutorials exist, but most of them are in French (see for example Perdoncin 2020). 

2. Several other metrics are produced such as the cos2—that gives the quality of the rep- 
resentation of a modality on the plane—and the v-test—a measure of association between 
variables. 

3. The number of axes that must be kept in the analysis is both a statistical and a scien- 
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as an opposition on the west-east axis (the x axis), and on the north-south 
axis (the y axis), with each part of the graph describing a subpopulation. It 
is often useful to exemplify these subpopulations with the description of the 
trajectory of one of the individuals that are positioned here. The following 
figure illustrates such results based on my research. Due to space restriction, 
I have limited the presentation of variables to those represented in the first 
two axes of the MCA, which gather most of the overall information. 

In figure 9, individuals (in grey) and the most contributing properties 
(in black) have been represented on the same graph. X axis explains 13.8 
percent of the total variance of the database, and the second one 8.9 per- 
cent. On the western part of the graph, one can see a set of properties that 
describe the deprivation of academic capital: people on this side of the space 
have signed the most recent OECD reports on health but have no PhD 
(“phd_no”) and no publication in central academic journals (“central_publi- 
”). On the opposite part of the graph (eastern part), people have published 
many articles in very central academic journals (“central_publi++”), are part 
of many journals’ reading committees (“reading _committee+”), and they 
generally hold a PhD in economics (“phd_eco”). They have signed the first 
OECD reports on health. 

Interpretation of the graph lies on the comparison of the subgroups con- 
structed by MCA. Variables that contribute the most to the construction of 
the axis oppose the two subpopulations. In this case, the first axis is mainly 
determined by the volume of academic capital retained by individuals. 
Thanks to MCA, one can see that individuals who signed the first OECD 
reports on health (in the eastern part of the graph) retain an important vol- 
ume of academic capital. This subpopulation is opposed to individuals who 
signed recent OECD reports on health and hold a much less important 
volume of academic capital. The method therefore shows that OECD has 
been legitimized as a central IO of the international field of health policy 
reforms thanks to enrollment of famous economists in its first reports. The 
description of the second opposition—north-south axis—refines the analy- 
sis. It must be read the same way as the first axis, using the most contributing 
variables to oppose the two polar subgroups. In this case, one can see that 
the second axis opposes internationally and nationally grounded trajectories. 
In other words, it opposes international and national civil servants. To sum 
up, MCA shows that the population of OECD reports on health signatories 


tific question. First axes explain more variance than the last ones. Most of the time, scholars 
keep in the analysis between two and five axes, the limit being the axis after which variance 
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Figure 9. Example of Multiple Correspondence Analysis 


is structured first by the volume of academic capital retained by the agents 
and second by the nationally grounded dimension of their trajectories. 
Furthermore, MCA is a powerful tool not only to describe a population 
but also to raise new research questions—for example here on legitimacy in 
the international field of health-care expertise, on the role of health econom- 
ics as a must-have curriculum for experts entering this field, or on the oppo- 


244 International Organizations and Research Methods 


sitions between national and international curricula to enter the field. MCA 
unveils the most structuring variables in the space and may therefore lead to 
new research questions, for example here on the history of health economics 
as a discipline, the universities where it has been developed, its appropriation 
by political agents, the importance of international trajectories to become an 
international expert on health care, and so on. 

‘The example reported here shows that MCA is even more useful when 
used in a multimethod research design (Martin-Mazé 2018). Archival work, 
interviews, and ethnography are often usefully combined with MCA: they 
first help researchers in deciding which variables to keep for the analysis, but 
they also allow researchers to relate the positions of individuals as objectified 
by MCA to positions they have taken, be they in interviews, observations, 
or paper-notes that would be stored in the archives. In the case of OECD 
health experts, observation of meetings where several report signatories were 
present allowed researchers to relate their interactions to the positions of 
individuals in the space designed through MCA. 


What Challenges? 


The main challenge of MCA lies both in the construction of the database 
and the data collection. Most of the time, for individuals, data is collected 
on the Internet, or through interviews and archives, but with potential biases 
that must be taken into account, and missing data. R package FactoMineR 
presents valuable tools to manage such biases. That said, the main challenge 
stems from the construction of the categories retained in the database that 
must be carefully justified. 

Once the MCA has been performed, interpretation of each axis may 
be tricky. It requires a good knowledge of the population, of the variables 
taken into account in the database, and a careful reading of the plot. Finally, 
one must note that statistical software to conduct MCAs can be quite dif- 
ficult to handle. Out of the geometric part of analysis, various metrics are 
produced by MCA (cos2, v.test, etc.) that require a certain level of statisti- 
cal knowledge to be explained. The open-source software R has a dedicated 
package called FactoMineR that comes with many handy tutorials and an 
active online community. 
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BOX U 


Building Databases on Individuals 


Kseniya Oksamytna 


Data on individual IO officials are important for researchers investigating 
issues like the political economy of IOs, autonomy and expertise of interna- 
tional bureaucracies, and IO performance and accountability. However, not 
all IOs provide comprehensive information on all relevant characteristics 
of their employees. When such data exist, they are often limited in scope 
(see box t—Navigating Human Resource Statistics). Researchers should care- 
fully consider whether building their own database of individual IO officials’ 
characteristics is worth the investment of resources. 

Building a database of IO officials’ characteristics should begin with an 
assessment of the existing data or ongoing data collection efforts by other 
researchers. If the data are indeed unavailable, the next step would typically 
involve a pilot project to determine the viability of the approach and the 
scope of the data. Researchers might reassess the project’s viability if they 
discover that information is classified, incomplete, hard to find, or difficult 
to categorize. If the pilot is successful, comprehensive data collection can 
begin. Since information on IO officials’ characteristics usually requires data 
collection rather than interpretation, intercoder reliability is not a significant 
issue, but members of the research team need to be on the same page about 
what information should be recorded and in what format. 

For example, in UN peacekeeping, data on the nationality, gender, and 
fatalities of military peacekeepers are readily available. There is less informa- 
tion on civilian peacekeeping officials or senior mission leaders, so Bove, 
Ruggeri, and Ruffa (2020) built a database of the characteristics of civil- 
ian and military leaders of UN peacekeeping operations, such as national- 
ity, tenure, and prior experience in a similar post. Their experience dem- 
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onstrates the advantages and costs of this approach: building the database 
of UN peacekeeping leaders entailed reading numerous UN press releases 
announcing leaders’ appointments, official secretary-general’s letters, and 
media reports for recording leaders’ names, nationalities, and start and end 
months of service. 

In deciding whether to construct a new database, researchers should weigh 
potential payoffs against the investment of resources. The main advantage 
of constructing such databases is that they allow systematic research into 
questions that could previously be examined only through single or com- 
parative case studies. The creation of a database on peacekeeping leaders, 
for example, has enabled an analysis of the role of cultural distance between 
peacekeeping leaders on the one hand and troops or the population on the 
other hand (Bove, Ruggeri, and Ruffa 2020), factors that affect peacekeep- 
ing leaders’ appointments (Oksamytna, Bove, and Lundgren 2021), and the 
UN Secretariat's efforts to hold leaders accountable for underperformance 
(Lundgren, Oksamytna, and Bove 2022). 

In the literature on international financial institutions, there is also a 
growing attention to IO officials’ characteristics that can be coded from 
project reports or LinkedIn profiles (Clark and Zucker 2022; Heinzel 2022). 
In summary, databases of IO officials’ characteristics can produce valuable 
insights, but require long-term planning and well-resourced teams. 
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CHAPTER 19 


Social Network Analysis 


Anna-Luise Chané 


Social network analysis (SNA) provides researchers with a toolbox to study 
relations or interactions between entities. Rather than examining the indi- 
vidual attributes of an actor, network analysis deals with the connections 
that exist between actors. It thereby offers a systematic approach for the 
analysis of relational patterns that shape the internal dynamics of interna- 
tional organizations (IOs), as well as their engagement with external actors. 


What? 


As the “science of interactions” (Maoz 2011: 6), SNA is particularly well- 
suited for the study of IOs, both with regard to their internal structures and 
their embeddedness in external networks. After early applications in sociol- 
ogy and anthropology, network analysis grew popular in the late 1960s and 
early 1970s across a broad range of disciplines. In international relations, it 
only began to appear in the late 1990s (Hafner-Burton et al. 2009), reflect- 
ing an increased focus on cooperation and networks. This development has 
been accelerated in the digital age by the growing availability of online data- 
bases and archival sources and the skyrocketing computing power of modern 
computers, which today allow researchers to store and analyze vast datasets. 

SNA is based on the premise that actors are not detached from their 
social context, nor solely driven by their individual characteristics, autono- 
mous from the influence of other actors. Instead it assumes that actors are 
embedded in social systems, which link them to other entities, and which 
can determine, enable, and restrict their behavior (Knoke and Yang 2020). 
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SNA seeks to measure and map the structures that emerge from these pat- 
terns of relationships, contacts, and interactions. It aims to explain how 
these structures develop over time, and how they influence the behavior of 
the network members. 

Consequently, researchers increasingly rely on SNA to explore, for exam- 
ple, membership patterns of IOs, the effect of IO membership on the behav- 
ior of states, the internal structures of IOs, and interorganizational relations. 
In the 1960s, Brams already drew on joint IO membership as one of the 
“flows of transactions” that defined the clusters of states in the international 
system (1966). Network analysis has since frequently been employed to 
map network structures in specific policy areas (Mérand, Hofmann, and 
Irondelle 2011) or IOs (Macon, Mucha, and Porter 2012; Dijkhuizen and 
Onderco 2019). The entities investigated may be states or groups of states 
but also individual persons. Several studies have examined the effects of IO 
membership, namely with regard to climate change (von Stein 2008) or con- 
flict (Hafner-Burton and Montgomery 2006; Dorussen and Ward 2008). 
Other studies have focused on interorganizational relations, exploring the 
position of IOs within the broader global network (Kim and Barnett 2000). 


Why? 


SNA offers researchers a powerful toolbox to systematically analyze rela- 
tional data and thereby obtain information about IOs that could not be 
gained by focusing only on the individual attributes of entities. A study of 
an IO could, for example, take stock of the various properties of its member 
states, such as their geographic location, wealth, and political ideology. Yet 
neither of these attributes would capture the complexity of the relations 
between the member states in the different bodies of the IO and therefore 
only give a partial view of the actual dynamics that shape the work of the 
organization. A network approach can complement the analysis by provid- 
ing information about the characteristics of the network as a whole, the 
positions of individual member states, and the existence and properties of 
subgroups. 

SNA therefore brings several advantages to the study of IOs. It captures 
the web of structural relations that are often more important for explain- 
ing an actor’s actions than their individual attributes (Hafner-Burton and 
Montgomery 2006). It takes into account both how an actor shapes a net- 
work through their behavior and how the network impacts on that behavior 
in return (structure and agency, Schulze and Ries 2017). And it allows for 
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the seamless bridging of several levels of analysis: from the individual entity, 
to a group of actors, to the whole network (Schulze and Ries 2017; Hafner- 
Burton et al. 2009). 

For example, it can be used for an initial mapping of an IO or a policy 
field, allowing the researcher to identify the most relevant actors or sub- 
groups, which could in a second step be explored through in-depth case 
studies (Chané and Sharma 2017). It can yield new insights about the pro- 
cesses that shape the development of international norms, by investigating 
network structures in multilateral fora (e.g., voting or sponsorship net- 
works). Unpacking the internal structures of an IO (as a network of mem- 
ber states, secretariats, and departments) also helps explore questions about 
organizational forms, and the challenges and opportunities they offer. SNA 
can reveal the dynamics that influence the legal interpretation of interna- 
tional norms, by exploring the interactions between courts. Furthermore, 
it can help capture the influence of IOs in policy networks, for example by 
assessing their relative position in issue-specific communication networks. 


How? 


At its most basic, a network consists of two elements: entities (nodes) and 
their relations with each other (ties). A node can be anything, from an indi- 
vidual person to an informal group, a state, an IO, or even an inanimate 
object, such as a judgment or a treaty. Similarly, a tie can be established 
by any form of contact between those entities, for example membership in 
an IO or ratification of an international treaty, participation in an event, 
information exchange, diplomatic relations, or armed conflicts. The type 
of relation on which data will be collected must be defined from the outset 
because it will determine the type of network that will be analyzed. Each 
entity will likely be involved in a myriad of different networks. For example, 
a state’s trade network will differ from its diplomatic or its development aid 
network. It is therefore important to carefully define the rules that deter- 
mine tie formation between nodes, and to clearly establish the significance 
of the selected relation for the research question. Joint membership in an 
IO, for example, is a widely used network variable, yet it is often insufficient 
to capture relations between member states. Multilateral fora may provide 
opportunities for engagement between their members, but the degree of 
actual interaction will depend on the setting. Council meetings in the EU, 
for instance, foster close exchanges among high-level politicians and will 
therefore build more influential ties than the meetings of “midlevel bureau- 
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crats in the African Groundnut Council” (Hafner-Burton and Montgomery 
2009: 579). Focusing on direct interactions, such as the joint tabling of 
proposals or joint organization of relevant events, may therefore yield more 
accurate results. 

Networks can take different forms. A relational (one-mode) network cap- 
tures direct relationships between two entities (e.g., trade, diplomatic rep- 
resentation), whereas an affiliation (two-mode) network captures common 
affiliation with a third entity (e.g., membership in an IO, participation in an 
event). A distinction must also be made between symmetric and asymmetric 
networks. A symmetric tie has equal value for both connected nodes (e.g., 
cosponsorship networks: state A and state B have both sponsored the draft), 
whereas an asymmetric tie does not (e.g., citation networks: judgment B 
refers to judgment A but judgment A, preceding judgment B, cannot pos- 
sibly cite judgment B). Researchers must also decide whether to adopt a 
whole network or an egocentric network approach. The former examines the 
relations between all nodes in a defined social setting, the latter focuses on 
one entity and studies its direct relations with other entities, as well as the 
relations between them. For example, a study of a specific policy area could 
include all IOs active in this field or select one of them and investigate its 
network relations. 

Once the research parameters are set, researchers must decide on the 
data collection method. In the early days of SNA, observational research 
was widely used. In IO research, this method offers the advantage of an 
insider’s perspective (see chapter 2—Participant Observation) and thereby 
may allow capturing a specific form of relation that would remain invisible 
through other means. At the same time, the difficulty of gaining access to 
what are often confidential settings, and the considerable time investment 
that is required, may pose challenges. In addition, given the size of the 
social setting under investigation, it may often be impossible to capture 
all relevant contacts, leading to gaps in the data, which are particularly 
detrimental in network analysis. Survey research offers a viable alterna- 
tive, though it also comes with challenges (see chapter 4—Surveys) (Scott 
2017). In particular, researchers must decide whether they will let their 
respondents choose possible answers from a predefined list or let them 
answer freely. The former requires that the researcher has identified all pos- 
sible entities in a network in advance, in order to ensure the completeness 
of the data. It can thereby minimize data gaps but may at the same time be 
cumbersome in cases of large networks with many possible responses. The 
second approach defines the network based on the responses by the survey 
participants. This may be useful in order to identify previously unknown 
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[ A C B 


Figure 10. Sample network graphs 


entities in the network, but it risks data gaps if responses are incomplete. 
The most convenient form of data collection will often be documentary 
research (see chapter 10—Document Analysis: A Praxiographic Approach). 
Today, most IOs make huge quantities of data freely available online, 
providing researchers with an easily accessible goldmine of information, 
including meeting and voting records, preparatory documents, activity 
reports, and treaty and case-law databases. 

Depending on the research objectives, the subsequent data analysis can 
focus on the overall network, on the position of entities within the net- 
work, on subgroups, or on a combination thereof. Network level measures 
include for example density and centralization, the former indicating the 
overall level of connections between the nodes, the latter the extent to which 
a network is focused on specific nodes. Figure 10 proposes sample network 
graphs showing (1) a centralized, sparse network, (2) a decentralized, dense 
network, and (3) a fragmented network. Actor-level measures determine the 
centrality of nodes in the network. They include degree centrality (the sum 
of the ties between the node and all other nodes), closeness centrality (the 
distance of a node to all other nodes in the network), betweenness centrality 
(the number of shortest paths that go through a particular node), and eigen- 
vector centrality (which also considers the centrality of the nodes with which 
a node is connected). Finally, SNA allows the identification of subgroups— 
clusters of more than three nodes that, for example, are all interconnected 
by ties of a certain strength (clique) or that are structurally similar (block or 
structurally equivalent cluster). 

‘The results may give insights about the efficiency and robustness of the 
network, about the level of coordination in subgroups (Dijkhuizen and 
Onderco 2019), and about the power of individual nodes. For example, 
nodes with a high degree centrality (such as nodes A and B in example 4) 
may wield social power and play an influential role in the network, whereas 
nodes with a high betweenness centrality may act as brokers, building bridges 
to less connected parts of the network (such as node C in example 4). 
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What Challenges? 


Despite the potential of SNA, researchers must also be aware of its limita- 
tions and challenges. Some of these are methodological in nature, such as the 
difficulty of dynamic network modeling, which captures the development of 
a network over time (Maoz 2011). Most studies offer static snapshots of a 
network, which yield no information about network growth or dissolution, 
about the changes in the positions of actors and subgroups. They may there- 
fore overlook crucial developments and ultimately offer a distorted view of 
the actual dynamics. 

Another challenge is multiplexity (Schulze and Ries 2017): networks 
with multiple types of ties between nodes. Entities are usually members of 
several networks, and the constraints they face in one network may impact 
on their behavior in another. A network based on one type of tie offers only 
a partial view of a more complex reality and may lead to erroneous con- 
clusions. To mitigate these shortcomings, it is often advisable to integrate 
SNA in a mixed-method research design, complementing it with statistical 
or qualitative approaches, in order to gain a deeper understanding of the 
individual attributes of the nodes, as well as the quality of their ties (Chane 
and Sharma 2017; Schulze and Ries 2017). 

In addition, researchers should be mindful of ethical concerns, which 
are particularly pressing when the entities analyzed are individual persons. 
SNA does not allow for the anonymity of the entities at the stages of data 
collection and analysis because the individual nodes and their relations need 
to be identified. It also does not allow for anonymous grouping of entities 
but identifies every node and its position in the network individually. Even 
anonymization of the final results will often not hinder the identification of 
the entities through reverse engineering. 

Finally, it is the highly technical character of SNA that constitutes per- 
haps the primary practical obstacle to a more widespread application in IO 
research. For those undaunted by this challenge, SNA offers a powerful tool- 
kit for the analysis of social processes. 


To Go Further 


Knoke, David, and Song Yang. 2020. Social Network Analysis. 3rd ed. London: SAGE 
Publications. 

Scott, John. 2017. Social Network Analysis. 4th ed. London: SAGE Publications. 

Scott, John, and Peter J. Carrington, eds. 2011. The SAGE Handbook of Social Network 
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INTERLUDE IV 


Challenging IOs through Numbers 


Simon Hug 


Introduction! 


My point of departure for this interlude is a simple one: there is nothing 
unique or distinctive to international organizations (IOs) that would justify 
specific methods. There might be specific issues in using particular methods 
when dealing with IOs (many of which appear more generally in interna- 
tional relations (IR) or political science scholarship), but I do not know of 
any specific method that is uniquely applicable to IO research, certainly not 
among quantitative methods. The chapters of this part of the edited book 
underline this quite convincingly as they discuss and illustrate (a subset of) 
methods widely used in political science and IR. 

Let me also be clear what I consider to be a method: a method is a set 
of procedures that allows us to adjudicate among rival hypotheses (Sprinz 
and Wolinsky-Nahmias 2004: 4). So a method is not a technique of data 
collection alone (e.g., participant observations, elite surveys, text or network 
analysis, etc.) nor is it a theory (see Sprinz and Wolinsky-Nahmias 2004 and 
Barkin in this volume, Interlude V—Controversies on Methodological Plural- 
ism, though this latter text unduly associates methods with epistemological 
stances).” Thus if one considers the use of quantitative methods in the study 
of IOs, one generally finds almost always related (though often delayed) pat- 


1. Helpful comments by Gerald Schneider and the editors of this volume are gratefully 
acknowledged. 

2. It is useful to note that in IR scholarship there is considerable confusion about methods 
and theoretical approaches (or, god forbid, paradigms; see, e.g., chapter headings in Reus- 
Smit and Snidal 2008). 
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terns as in IR or political science more broadly. Thus specific quantitative 
tools were introduced (or discarded) in political science or IR more gener- 
ally, before similar introductions (and eliminations) occurred in research on 
IOs. While Singer and Alger’s (1968: 3) quip that “[I]f it can be measured, 
it must be trivial” has been made to characterize the critique of quantitative 
work in IR more broadly, it certainly also characterized scholarship on IOs 
critical of quantitative methods more specifically. 


lOs, IR, and Numbers 


If one considers systematic work in IR, a rather schizophrenic picture 
emerges when it comes to quantitative studies. On the one hand, as Singer 
and Alger’s (1968) volume demonstrates, even in the late 1960s quantitative 
work still had to fight for being taken seriously in the broader IR scholarship. 
On the other, on some aspects of scholarship on IOs quantitative studies 
were almost the only game in town when it comes to systematic work. Thus 
Alger (1970) notes that in the 1950s and 1960s reputable journals, includ- 
ing International Organization, while focusing on IOs, often published few 
systematic studies. He also notes that a large share of these systematic studies 
analyzed quantitatively voting in various international assemblies (more on 
this below). 

‘Thus in some sense the IO literature followed a similar pattern as many 
other areas of IR scholarship, from mostly qualitative case studies to sys- 
tematic comparisons and finally studies taking inferential problems seri- 
ously. This is reflected in the increasing importance of quantitative work (see 
Sprinz and Wolinsky-Nahmias 2004: 4). Nevertheless, quantitative schol- 
ars in IR more generally and IOs (Hafner-Burton, von Stein, and Gartzke 
2008) more specifically still feel the need to defend their approaches.’ 


Examples of the Use of Quantitative Methods 
It is useful to discuss why methods used in research on IOs are hardly 


(if ever) different from methods used elsewhere in IR or political science 
research by focusing on some examples, which closely follow the chapters 


3. Note also, however, that some national, mostly European, literatures have dissociated 
themselves from international trends. 
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brought together in this part on quantitative methods. It bears noting that 
most of these chapters do not present and discuss “methods” in a strict sense 
but focus on measurement tools. 


Voting 


As the review of systematic work on IOs in the 1950s and 1960s by Alger 
(1970: 444) nicely shows, analyses of voting, mostly in the General Assem- 
bly of the UN (UNGA), dominated (see review offered by Voeten in chapter 


14— Voting Analysis)“ which led this author also to a critical assessment: 


On the other hand, given limited skills, time, and money it seems rea- 
sonable to conclude that too much of what is available has been put 
into voting studies. After all, roll-call votes cover only about one-fifth 
of the votes in the United Nations General Assembly, and more than 
half of the decisions are made without voting at all. 


Thus, as Alger (1970) states, most studies on voting in IOs neglect that 
much business is conducted in these assemblies without votes. Similarly, 
the process leading up to these votes (and their content) is almost always 
ignored (see relatedly Conrad and Monroe 2021), which is considerably 
problematic as numerous studies unreflectively use such voting records to 
provide measures of state preferences (see discussion in chapter 14— Voting 
Analysis), Thus a large part of the voting literature on IO reads as if very little 
had happened since Rice (1928) pleaded for quantitative analyses. Similarly, 
few studies explore tools developed in political science and related disci- 
plines to analyze voting records both to infer positions but also, at the same 
time, to test specific hypotheses, as done in Hug and Lukacs’s (2014) study 
of what explains votes at the UN Human Rights Council. 


4, The importance of voting in systematic quantitative studies of IOs is obviously reflective 
of the very early quantitative work on voting in parliaments initiated by among many others 
Rice (1928). 
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Individuals and MCA 


A look back at the early IO literature (see Alger 1970) suggests that a few 
systematic studies already early on focused on individuals. Thus Keohane 
(1969) relied on similar sources as those discussed by Oksamytna (box 
u—Building Databases on Individuals) and Badache (box t—Navigating 
Human Resource Statistics). In many traditions of IR research, however, 
such a focus on individuals appeared meaningless. Nevertheless, some path- 
breaking work on negotiators by Jacobson et al. (1983) illustrated how the 
behavioral revolution also was of relevance for IR scholars generally and 
IO researchers more specifically. As much of this individual-level data is 
quite plentiful, quantitative methods used in the social sciences experiencing 
the behavioral revolution were deployed to great effect. While much of this 
work adhered to an inferential strategy, in some areas (both topically and 
geographically) more descriptive tools were deployed, as for instance the 
tool discussed by Brissaud (chapter 18—Multiple Correspondence Analysis). 
For inferential, and thus mostly explanatory purposes, however, such tools 
are quite inadequate.’ 


Text and Social Network Analysis 


Needless to say, the analyses of texts and interactions in a quantitative way 
has featured prominently in the IO literature. Much of this work, however, 
as discussed by Mitrani and Noy (chapter 17— Computerized Text Analysis) 
and Chané (chapter 19—Social Network Analysis), aimed at generating mea- 
sures of particular features of IOs and thus offered basically tools for gen- 
erating information. Obviously in the field of network analyses, to already 
foreshadow the end of this part, methods have been developed that allow 
testing hypotheses and theories. Similarly, text analyses of the quantitative 
type can be integrated in a method to do the same. As the latter is hardly 
done in the literature on IOs, I will refrain from covering this second devel- 
opment, while returning to the first below. 

Thus Graber (1969) had recourse already to such a measurement strategy 
in order to assess the perceptions in the UN regarding the conflict in the 
Middle East. Doing so the author relied on practices and tools widely used 


5. Thus Brissaud’s (chapter 18—Multiple Correspondence Analysis) reference to hypotheses 
will strike many readers as odd, as he is quite explicit about the fact that MCA proceeds 
inductively. 
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in other fields, and thus it cannot surprise that Mitrani and Noy (chapter 
17—Computerized Text Analysis) in their review basically discuss tools used 
in text analysis in other fields of research. Also the three main challenges for 
using these tools in IO research that they identify, namely translation, con- 
text, and intention, are hardly specific to IOs (e.g., Proksch and Slapin 2012). 
Thus while there is interesting work on IO texts using automated text analysis, 
much of the work focusing on speeches does not yet sufficiently take the con- 
text and intentions in international assemblies and parliaments into account. 

Similarly, analyses of interactions have also played an important role in 
work on IOs since the early quantitative studies. Thus Alger (1966) carried 
out a detailed analysis of interactions among delegates in the 5th committee 
of the UNGA. Much work on interactions and networks reviewed by Chané 
(chapter 19—Social Network Analysis) is now commonly known as Social 
Network Analysis (SNA), which corresponds to a set of tools to describe 
interactions and network characteristics. Thus SNA used in this way is basi- 
cally a measurement tool that can be used to describe networks. Summaries 
thereof can be powerfully used in subsequent traditional quantitative analy- 
ses. More recently, however, several scholars rely on network analyses in an 
inferential, explanatory perspective, mostly drawing on Exponential Ran- 
dom Graph models. In this vein, Cranmer, Heinrich, and Desmarais (2014) 
consider how network effects play out in international economic sanctions, 
especially how reciprocity influences sanction regimes. 


Quantitative Analyses 


It is probably fitting to end this interlude with a section on quantitative 
analyses more broadly. As such analyses are broadly dominant in the IR 
literature, it is not surprising that they also have their place in IO research. 
Hence Roger’s (chapter 16—Large-N Data and Quantitative Analyses) review 
eloquently shows that research questions addressed in this field find almost 
always close parallels in other areas of political science research. Thus the 
problems of quantitative work on IOs this author highlights are not unique 
to this field but are largely identical to the ones present in quantitative IR 
and political science more broadly. 

‘These issues are also convincingly illustrated by the application proposed 
by Cottier and Bohnet (chapter 15—Statistical Analyses with IO Data). Data 
provided by IOs are often collected for very different purposes than those for 
which researchers want to use them (as in most subfields, see Lustick 1996), 
which also implies that researchers have to be careful how they are used. 
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Conclusion 


Systematic studies of IOs have, as Alger (1970) nicely demonstrates, already 
a considerable history. Methods used (often earlier) in IR and political sci- 
ence research found entrance in the field of IO research and have allowed for 
numerous novel insights. Thus quantitative work has gained in stature in IO 
research (as in IR more generally), but as illustrated by the discussion of the 
examples above, in many areas of IO research quantitative scholars neglect 
important caveats of not only quantitative but also qualitative research. 
Consequently, the use of voting records without considering what leads to 
votes (and how this changes over time), or considering documents from IOs 
uncritically is considerably problematic. Similarly, in assessing the conse- 
quences of IOs it is baffling that work still fails to consider that IO mem- 
bership is largely endogenous to many outcomes we might be interested in. 
These shortcomings, while easy to document and illustrate for quantitative 
research, are affecting, however, both qualitative and quantitative work on 


IOs. 
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PART 5 


Combining 


Introduction 
Combining 


Fanny Badache, Leah R. Kimber, and Lucile Maertens 


In order to answer a research question, scholars often rely on two or more 
methods introduced in the first four parts of this book to generate, collect, 
and analyze data. This section, entitled Combining, discusses the advantages 
and challenges of explicitly and purposely combining methods in research 
projects on IOs while providing practical examples. 

Historically, scholars have mostly limited their methodological toolkit 
to the methods associated with their discipline. Methods of observation, 
for instance, have originally been used by anthropologists and sociologists 
(see part 1—Obdserving), while archival research was taken on by histori- 
ans (see part 3—Documenting). However, in the last decades, scholars in 
social sciences and the humanities have increasingly mobilized an array of 
methods broadening the scope of their methodological approaches. Draw- 
ing on methods stemming from different disciplines has allowed scholarship 
in IO research to witness an increasing—and welcoming—interdisciplinary 
dialogue. 

‘This section shows how studying IOs by combining methods has mul- 
tiple advantages for doing empirical research in the ever-changing contexts 
of multilateralism. On the one hand, the use of multiple methods is relevant 
to address specific theoretical questions such as unpacking power relation- 
ships, going beyond institutional discourse, understanding the complex 
agency of IO actors, or tracing the origins of a particular idea or program. 
On the other hand, at a methodological level, researchers have suggested 
several rationales for combining methods. Being aware of the inherent limi- 
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tations of each method, researchers combine methods to overcome these 
in order to search for more validity (triangulation). Using various methods 
and thus intertwining data may come in handy to complement and expand 
the results derived from one method, which in turn allows researchers to 
explore the findings further. The combination of methods may at times also 
be needed to overcome the challenges researchers face while doing field- 
work, as it reveals the crafted nature of social science research (see chapter 
29—Composing Collages: Working at the Edge of Disciplinary Boundaries). 

This having been said, contributions also point to important challenges 
in combining methods. For instance, it supposes a high diversity of meth- 
odological skills and generally calls for more material resources. Last but not 
least, studies that combine methods are often longer both in the time dedi- 
cated to data generation and analysis as well as in their written format. The 
latter may collide with scientific journals that restrict the number of words 
in their publications. 

The contributions gathered in this section address the issue of meth- 
odological combination in three ways. First, scholars provide examples of 
the combination of some established methods in social sciences (see chapter 
20—Interviews and Observations, chapter 21—Observations, Interviews, and 
Archives, and chapter 22—Computational Text Analysis and Archival Meth- 
ods). They explore why and how these specific methods can be combined, 
what the value added is, and point to their reoccurring challenges. In box 
v—Challenging Secrecy, combining methods helps overcome the hurdles of 
secrecy in IOs for instance, while box w—Research with LinkedIn, shows 
how digital platforms provide complementary information. The possibility 
of combining methods is, of course, not limited to these examples. Second, 
contributors shed light on how combining methods can be done as part of a 
coherent research design (see chapter 23—Qualitative Comparative Analysis, 
chapter 24— Structured, Focused Comparison, and chapter 25—Process Trac- 
ing), guiding the reader on how to make sense of such combinations. Third, 
contributors present methodological approaches that in essence necessitate 
the combination of various methods for their data generation, data collec- 
tion, and data analysis (see chapter 26—Prosopography, chapter 27—Practice 
Analysis, chapter 283—Feminist Approaches, chapter 29— Composing Collages: 
Working at the Edge of Disciplinary Boundaries, box x—Postcolonial Insights, 
and box z—Expeditions as a Research Method). These contributions focus on 
a particular research approach and its adaptation to the IO context. 

In sum, the final part of this book opens with a reflection on methodol- 
ogy (see box y—Reflexivity in Practice), broadening the specific challenges 
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on methods per se. Taken together, these contributions shed light on three 
important dimensions when combining methods: the rationales, timing, 
and practical implications. While all show that combining methods allows 
researchers to grasp the heightened complexities of IOs, they also provide 
rich examples by advocating in favor of methodological pluralism in the 
study of IOs (Interlude V—Controversies on Methodological Pluralism). 


CHAPTER 20 


Interviews and Observations 


Kari De Pryck 


Who has never had, reading a report by an international organization (IO) 
or interviewing an international actor, the impression of being served the 
official discourse, a politically correct speech that shies away from sensitive 
and contentious issues? While this may not be a problem if you are interested 
in the representations and worldviews projected by IOs, it may become frus- 
trating when you seek to study what they do in practice. Researchers here are 
interested in daily activities, which are often more complex than what pro- 
cedures let transpire, loopholes in the procedures (which give bureaucrats 
room for maneuver), and cases of mismanagement (no one is perfect). Yet 
you will often be told, especially in times of crisis, that there is nothing more 
to say than what is already available in the official reports and press releases. 
In this chapter, I discuss how to combine interviews and participant obser- 
vation to circumvent (and at the same time study) IOs’ strategies to control 
the information that is being shared in official documents and discourses. 
Both methods can be used to generate original data on the day-to-day life 
of IOs and to complement other approaches that delve into the documents 
and reports that they produce. 


What? 


Interviews and observations are two well-established qualitative methods in 
the social sciences, which are often used to study IOs (see part 1— Observing 
and part 2—IJnterviewing). Interview research, the craft of collecting data 
through conversations (Kvale and Brinkmann 2009), may take different 
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forms depending on the research design: interviews may be structured (rely- 
ing on a strict set of questions), semistructured (relying on an open set of 
questions) (see chapter 5—Semistructured Interviews), or unstructured (with 
no predetermined questions). They also often have different purposes. For 
instance, ethnographic interviews (see chapter 3—Ethnographic Interviews) 
aim at learning about a given culture from the interviewees’ perspective. 
Biographical interviews (see chapter 6—Biographic Interviews) seek to gain 
access to the life histories and career trajectories of individuals. Finally, prax- 
iographic interviews (Bueger and Gadinger 2018) emphasize collecting clues 
about the practical knowledge of participants. The data takes the form of 
interview notes, recordings, and transcripts. 

Instead of experiencing IOs through the eyes of an informant, direct 
and participant observation seeks to obtain an unmediated personal wit- 
nessing of them. Such a method (see chapter 1—Direct Observation and 
chapter 2—Participant Observation) ranges on a continuum from passive 
(involving no or marginal intervention from the researcher into the activi- 
ties under study) to active participation (including engaged participation by 
the researcher in those activities). The researcher may occupy different roles 
in an organization, as a member of a government delegation, as an observer, 
or as a staff at headquarters, regional, or national offices. The data takes 
the form of fieldnotes and other documents gathered during the immersion 
period. Depending on the role, the sensitivity of the data may be more or 
less important—for example, an observer will be less confronted with confi- 
dentiality issues than a member of a national delegation or a staff. 

Combining interviews with observations may provide rich and detailed 
insight into the depth of IOs and the individuals who gravitate around 
them. Both methods are often used in ethnographic research (Spadley 
1979) and in mixed-method approaches, which combine both qualitative 
and quantitative data collection and analysis (Small 2011). For instance, 
Lucile Maertens and Raphaélle Parizet (2017) used participant observa- 
tion and semidirected interviews (in addition to document analysis) to 
compare depoliticization practices in the United Nations Environment 
Programme's Post-Conflict and Disaster Management Branch and the 
United Nations Development Programmes Country Office in Mexico. 
Both methods are used to complement each other and gain the widest per- 
spective on those organizations. In my own work on the Intergovernmen- 
tal Panel on Climate Change (IPCC), I combined ethnographic meth- 
ods with descriptive statistics (Gros 2017) and used a database of IPCC 
scientists and diplomats to quantify insights from direct observation and 
interviews (De Pryck 2022). 
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The most obvious advantages of combining multiple methods is that the 
findings can be used to confirm and/or complement each other: the data 
derived from one method can be used to verify findings derived from the 
other and/or compensate for its weaknesses. Because different types of data 
generally produce different types of knowledge, complementary research 
design is more frequent than a confirmatory one (Small 2011). 

Combining interviews and observations is particularly relevant to enlarge 
our understanding of IOs beyond what is publicly communicated in official 
documents. For instance, observations allow researchers to experience an IO 
through their own eyes, adding to the testimonies gathered in the interviews. 

In the first stage of my research, direct observations of IPCC plenary 
sessions (where the member states of the organization meet) were pivotal 
to develop a familiarity with the inner workings of the organization that I 
would not have gained if I had only lived it through participants’ accounts. 
Such familiarity also contributed to enhance the relevance and depth of my 
interviews (because I had gained inside knowledge). On the other hand, 
interviews are particularly informative, at a second stage, to reflect on the 
data collected during the observations. Because the experience of the embed- 
ded researchers is often bound to the position they occupy (from observer to 
staff or delegate), the accreditation they receive (complete or limited access), 
and the length of the immersion (from a few weeks to a few months/years), 
interviews can be useful to “fill in the blanks” of observations. For instance, 
in my research on the IPCC, I conducted several formal interviews with 
informants that I had met during my fieldwork to gather their perspectives 
on certain outcomes and to help me put my observations into perspective. 
Finally, interviews are also relevant to confirm information that, for confi- 
dentiality reasons, cannot be displayed as excerpts from the field. 

Another major advantage of combining interviews and participant obser- 
vation is that it makes it possible to circumvent (and study) the strategies 
of stage management by IOs. Stage management is a concept introduced by 
the sociologist of science Stephen Hilgartner to describe the “techniques for 
controlling what is publicly displayed and what is concealed” by an institu- 
tion (Hilgartner 2000: 7). While originally introduced to study the pro- 
duction of science advice, stage management strategies are also visible in 
the practices of formal organizations (Broadhead and Rist 1979). As noted 
by Hilgartner, “bureaucracies—well known for their elaborate external and 
internal boundaries—typically segregate audiences and separate informa- 
tion into multiple regions, carefully controlling access to written material” 
(2001: 19). Drawing on Goffman (1959), Hilgartner distinguishes between 
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the front stage (the official views and images communicated to external audi- 
ences) and the backstage (the information and practices that are concealed 
to them). International bureaucracies employ very similar strategies, care- 
fully selecting the kind of information that is publicly displayed, the indi- 
vidual entitled to speak on their behalf, and under which conditions. In this 
context, interviews and participant observation can be useful to observe the 
front and back stages of IOs. Interviews were for instance very relevant to 
reconstruct the crisis that shook the IPCC in 2009, when errors were found 
in its Fourth Assessment Report—for instance, regarding a rapid retreat of 
Himalayan glaciers. The organization responded officially only a month 
later (after a series of blunders by its chair, which included comparing cri- 
tiques to “voodoo science”). Interviews allowed me to better understand the 
circumstances that led to such delay in responding to critiques: the IPCC 
secretariat had been under a lot of pressure following growing media atten- 
tion on climate change and did not have a proper communication strategy. 
Besides, members of the Bureau (the governing body of the IPCC) did not 
agree on who should take responsibility for the errors (De Pryck 2021). 


How? 


When using interviews and participant observation, a number of elements 
should be considered at different stages in the research. I briefly mention a 
few, for they are already addressed in greater details in parts 1 and 2 of this 
volume. 

Since different methodologies lead to different types of research ques- 
tions and hypotheses, it is important to reflect on the scope of the research 
early on and be explicit about the rationale for the selected methods. While 
this is true for all scientific investigation, it is particularly important when 
combining methods as this requires collecting data in ways that facilitate 
their combination at a later stage. Will the interviews be conducted before, 
during, or after participant observation? Will the interviews seek to reflect 
on findings stemming from the observation or explore issues that were not 
covered in the observation, or both? The combination of methods in time 
and space needs to be well thought out. 

In the context of increased attention to data protection, researchers are 
generally requested to provide interviewees with a formal document explain- 
ing the objective of the research, the terms of the interview (leaving to the 
interviewee the possibility to decide if they want the interview to be recorded 
and if their identity should be anonymized), and the use that will be made 
of the data. This procedure is also a way to reassure the informants that their 
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data will be used appropriately and to convince them of the legitimacy of 
the research. Information on data protection must in some cases also be pro- 
vided when carrying out participant observation—for example, how will the 
identity of participants be protected? Will recording be used? Universities 
are increasingly requiring researchers to win approval by their ethical review 
boards to carry out research. 

When preparing for interviews, researchers need to identify the partici- 
pants who will be included in the study and on which ground: employees 
of a particular department and/or program? Government delegates? Third 
parties such as civil society groups? In certain cases, the geographical repre- 
sentativeness of the participants needs to be considered to avoid perspectives 
being underrepresented (e.g., from the Global South). Additional questions 
include how the potential interviewees will be contacted (by email or on site) 
and how the project will be introduced to them. Having inside knowledge 
of the IO’s functioning can be an advantage when making a request for an 
interview because the researcher may be seen as an “insider’—interviewees 
may feel more at ease and share more stories than with an “outsider.” On the 
other hand, it can be a disadvantage especially when the interviewee seizes 
the opportunity during the meeting to gain insight from the researcher, who 
in turn runs the risk of losing the thread of the discussion to the extent of 
becoming themselves the interviewee. 

When planning participant observation, the researcher needs to con- 
sider the different positions that can be occupied in an IO, because each 
role generates different types of knowledge. For instance, it is not the same 
to observe diplomatic deliberations from the standpoint of a civil society 
group/observer, of an IO employee, or of a government delegation (see table 
3 in chapter 2—Participant Observation). In multilateral forums, the accred- 
itation also delimits the spaces to which researchers have access. Observers 
or third parties for instance can attend the plenary room, where the general 
debate takes place, but they rarely have access to breakout rooms (where 
governments negotiate specific issues behind closed doors). Depending on 
access rights and responsibilities, interviews can be used to gain information 
on the unfolding of meetings that the researcher could not cover due to 
limited access. 


What Challenges? 


Both methods have their limitations, some of which have been already 
mentioned (the question of recruiting informants, of getting access to IOs, 


Interviews and Observations 273 


and so on). To avoid them, researchers can rely on other sources of data to 
complement and contrast their findings. IOs produce a large number of 
documents (procedures, minutes, guidelines, reports, press releases, and so 
on) and make them available on their website or in their archives. Using 
this data can be very useful to put observations into perspective and to sup- 
port sensitive findings collected through participant observation with other 
documents. In my research on the IPCC, I also made abundant use of the 
participants’ comments that I could find in books and in scientific journals 
(e.g., Nature and Science). 

While combining interview and participant observation is relevant to 
explore the backstage of IOs, both methods can be subject to stage manage- 
ment strategies and have their limitation, even combined. Interviews can 
become performances, during which the interviewee subtly emphasizes cer- 
tain issues while concealing or remaining vague about others. Interviewees, 
and in particular scientific or technical experts, may use techniques of bound- 
ary work (Gieryn 1983) by seeking to present their activities as objective and 
nonpolitical. Such strategies are also prevalent in IOs (Louis and Maertens 
2021). In my research on the IPCC, I soon realized that some of the inter- 
viewees were giving me a “guided tour” of the organization and avoided 
sharing information about the internal dynamics of the assessment process. 
I realized that this type of discourse reflected a more generalized strategy of 
the organization that consisted in emphasizing the procedures guiding the 
assessment, while remaining discreet about the unwritten rules that shape 
the internal deliberations between scientists and diplomats. While partici- 
pant observation allows researchers to bypass such strategy, certain arenas 
will still be difficult to access or remain closed to researchers (see, e.g., Bour- 
rier 2017 on getting access to the World Health Organization). 

In a context where practices of IOs are increasingly scrutinized, stage 
management strategies have become more common (e.g., employees receive 
media training and are briefed on how to respond to sensitive questions; 
access to researchers is made more difficult and bureaucratic) to the point of 
potentially creating blind spots that cannot be filled by any research method. 
Taking stock of these hurdles leads researchers to recognize that social reality 
is always more complex than what research methods allow them to see. 


To Go Further 


Bueger, Christian, and Frank Gadinger. 2014. International Practice Theory: New Per- 
spectives. London: Palgrave Macmillan. 
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Hilgartner, Stephen. 2000. Science on Stage: Expert Advice as Public Drama. Palo Alto: 
Stanford University Press. 

Müller, Birgit. 2013. The Gloss of Harmony: The Politics of Policy-Making in Multilateral 
Organisations. London: Pluto Press. 
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BOX V 


Challenging Secrecy 


Olivier Schmitt 


Accessing data in an international organization (IO) may be challeng- 
ing because of the secrecy in which policymaking activities are sometimes 
shrouded. Secrecy is here understood as the institutional process of conceal- 
ing information about the organization, for operational and security pur- 
poses. This concealment happens through work cultures (some organiza- 
tions are more or less open about their activities) and/or through specific 
regulations. Challenging secrecy is then about knowing how to uncover 
secrets, determining which are relevant, and exercising judgment in publish- 
ing these secrets. It is a well-known fact that bureaucratic organizations tend 
to overclassify information, and IOs are no exception to the rule. ‘This is 
particularly acute in security-oriented IOs such as the North Atlantic Treaty 
rganization (NATO) or the Organization for Security and Cooperation in 
Europe (OSCE), but lifting the veil on particular activities is a challenge that 
researchers working on any IO will encounter. 

A second issue to keep in mind is that unveiling secrets is different from 
getting a “scoop.” To answer the vast majority of research questions, under- 
standing social patterns (such as decision-making processes) is far more 
important than obtaining very specific information such as the precise 
wording of a classified document. Fortunately for researchers, the former is 
usually more easily done than the latter. Work practices and social patterns 
can be secret (in the sense that they are not openly exposed outside of the 
organization), but they are not necessarily protected (in the sense that their 
access is restricted according to a specific set of regulations) the way specific 
documents usually are. 

‘The third issue to address is the ethical dimension of unveiling secrets. 
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Peoples’ careers and lives are on the line, and the commitment to research 
needs to be bounded by the principle of “first, do no harm.” The very 
unlikely situation of uncovering blatantly illegal activities aside, researchers 
need to assess the data they publish, taking into consideration the impact 
on people inside the IO under study: it is very possible that the researcher 
uncovers more than what is necessary to make a compelling argument, and 
not everything unveiled during the research process needs to be published. 

With these three issues in mind, different methods facilitate the unveil- 
ing of secrets within IOs. Participant observation is obviously an effective 
way to circumvent secrecy, since the researcher is embedded within the orga- 
nization (see chapter 2—Participant Observation). Some form of nondis- 
closure agreement should be agreed upon in advance in order to delineate 
the boundaries of the final publication. Participant observation can be dif- 
ficult to establish, and interviews are another way for researchers to access 
relevant (secret) data (see chapter 5—Semistructured Interviews and chap- 
ter 6—Biographic Interviews). The methodological principles discussed in 
chapter 3 (Ethnographic Interviews) are of course applicable here, but it is 
worth mentioning that, in my experience, interviewees are more comfort- 
able discussing sensitive aspects of their work when note-taking is preferred 
to recording. Finally, it is important to protect the data obtained in line with 
each university's ethical requirements. 
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CHAPTER 21 


Observations, Interviews, and Archives 


Marieke Louis 


“Like ranchers and farmers, historians and social scientists lay claim 
to the same terrain with different purposes in mind. . . . Fortunately, 
the conflict between the two professions has never gone beyond 
rhetorical violence, but it has at times been acute and, in my opinion, 
counterproductive.” (Lebow 2001: 111) 


The question of mixing methods like observation, interviews, and archives 
analysis is closely intertwined with the debates on interdisciplinarity in social 
sciences and distinctions going far back into the nineteenth century, essen- 
tially between anthropology and ethnography, and sociology and history 
(Arborio et al. 2008; Cohen 2008; Bloch 1992; Buton and Mariot 2009; 
Müller 2006; Weber 1996). While these disciplinary borders might be com- 
forting, many in the IR field admit that building bridges between disciplines 
and research traditions is a more fruitful way to study not only international 
politics in general (Elman and Fendius Elman 2001) but also IOs in both 
past and present times. 


What? 


In this chapter, observation goes from rather passive activities like attending, 
reporting, recording, and taking pictures during international meetings as 
well as witnessing ordinary interactions during more informal situations, 
to more active forms of observation pertaining to participant observation 
(internship, expertise, and so on) (see chapter 1—Direct Observation, chap- 
ter 2—Participant Observation, and Chapter 20—Interviews and Observa- 
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tions). Interviews mostly encompass the techniques of qualitative semistruc- 
tured discussions lasting from thirty minutes to two hours or more with a 
variety of actors (diplomats, national or international civil servants from 
director to lower-level staff, trade unionists, employers’ representatives, 
NGO members, and so on) (see chapter 3—Ethnographic Interviews and 
chapter 5—Semistructured Interviews). Finally archive analysis relates to the 
examination of two main kinds of sources: the consultation of documents 
(physical or online) stored by and within the archive department of the 
organization under study (if existent), but also “self-constituted” archives, 
meaning documents related to past events that are not necessarily considered 
worthy of attention by the institution and thus not stored within IOs by 
professionals (see chapter 8—Archives). They can either be collected during 
fieldwork or shared by private actors during the research process. While only 
the former might be officially stamped as archives, the latter also deserves 
our attention as such sources will be examined and used in similar ways as 
official archives. 


Why? 


Combining the study of archives with observation of past and present situa- 
tions as well as interacting—mainly through interviews—with contemporary 
individuals who shape or have shaped the everyday life of IOs is not just 
a tool for fact-checking and triangulation. It also allows the researcher to 
achieve more robust results and a deeper understanding of a given problem 
such as representativeness claims at the ILO (Louis 2016), gender within 
1O-led international programs (Saiget 2017), food security at the FAO 
(Cornilleau 2019), heritage protection during UN peacekeeping operations 
(Leloup 2021), among others. Such combination is particularly relevant, 
even necessary, when the researcher is confronted with the study of change 
and continuity, legitimization processes, institutional mechanisms, and 
routines firmly rooted in the past and the tradition of an organization like 
tripartite representation in the ILO (Louis 2016) or the influence of neoclas- 
sical economics at the FAO (Cornilleau 2019). For each of these topics, the 
mix of methods performs three distinct though complementary functions: 
informing, confronting, and remembering. 


(1) The study of change and continuity over a long period of time 
is the most obvious reason to combine observation, interviews, 
and archive analysis. As most of the IOs we study nowadays 
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were created at the end of the nineteenth and beginning of the 
twentieth centuries, we are confronted with a problem of miss- 
ing sources related to both past and present facts. In some cases, 
archives are not accessible for reasons of confidentiality, disclo- 
sure rules, loss during IOs’ moves, thereby rendering the resort to 
interviews and, when possible, oral history even more necessary. 
For instance, in the case of the International Chamber of Com- 
merce (created in 1919) many archives were either lost or only 
episodically stored. Sometimes key individuals did not leave any 
archival record at all. Interviewing them (or their colleagues or 
relatives if deceased) might therefore be a way to circumvent the 
dearth of empirical material whether we are interested in past 
or present events. Reciprocally, when one intends to go back to 
the roots of a specific issue, interviews and observation are not 
always adequate sources, as events might go way back beyond 
what contemporary individuals can actually know or remem- 
ber. Archives are therefore necessary to reconstruct the genesis 
of the problem under study. Without considering observation, 
interviews, and archives as substitutable sources, one can think 
of combining them as a way to reconstitute the whole picture of 
the problem under study and to fill out empirical loopholes (see 
chapter 20—ZJnterviews and Observations). Moreover, in order 
to claim any kind of “turning point” within IOs, we need to be 
able to document a “before” and an “after” without consider- 
ing that these categories are self-evident and mere chronological 
attributes. The more documented these historical periods are, 
the less exposed to the criticism of restrictive case selection the 
researcher is. 


Combining observation, interviews, and archives is also useful in 
order to challenge dominant discourses produced by IOs, as insti- 
tutions often prone to consensus and reluctant to publicizing con- 
troversies and conflicts, especially when these discourses rely upon 
convictions and narratives deeply rooted in the past to the extent 
of being almost naturalized and internalized by institutional actors. 
Combining methods therefore aims at circumventing silences or 
establishing critical distance from subjective and sometimes dis- 
torted perceptions of reality. The idea is thus to confront discourses 
with practices in very concrete terms and not just abstract hypoth- 
eses. Discourses can be documented either through observation, 
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interviews, or archives. In order to challenge legitimization strate- 
gies though, the purpose of combination is to confront discourses 
(through interviews or archived speeches for instance) with actual 
practices collected by the researcher during their observation or 
archival investigation. Combining methods can therefore prove 
useful in order to highlight the dominant discourse within an 
institution, but also to explore alternative narratives and institu- 
tional options that have been either left aside or silenced by spe- 
cific, often dominant, actors (Louis 2016; Cornilleau 2019). This 
aspect has been well demonstrated by critical (mostly feminist and 
postcolonial) studies (Aradau and Huysmans 2013; Decker 2013; 
see chapter 28—Feminist Approaches). In some cases, this combi- 
nation might even raise the awareness of actors involved within 
IOs toward their own processes and eventually have a transforma- 
tive impact on them. Finally, relying upon archives and previous 
observation of specific situations is also a good way to assert the 
legitimacy of the researcher (Laurens 2007: 117), especially when 
facing interviewees who are reluctant to social sciences methods of 
investigation and try to dismiss some findings as irrelevant or anec- 
dotical. As emphasized by Sgard (2019), combining archival work 
and interviews on the International Monetary Fund, interviews 
must necessarily complement archives if one wants to document 
controversial phenomena such as conflicts of interests or collective 
deficiencies. 


(3) Beyond challenging dominant discourses, the purpose of com- 


bining methods is also to overcome institutional silences and 
memory loss: it can refer to inconvenient truths that have been 
intentionally “buried” but also unintentionally forgotten events 
or processes (Louis and Maertens 2021). This aspect of institu- 
tional (loss of) memory is especially relevant when analyzing IOs 
that have accumulated a considerable amount of documents, 
sometimes over more than a century, and encountered a high staff 
turnover. In the case of memory loss, the lack of archives can be 
circumvented by an oral testimony or a direct observation. Like- 
wise a deficient individual memory can be compensated by avail- 
able archives or direct observation of the situation under study 
(Louis 2016). 


How? 


I hereafter describe three cases drawn from my own research on the Inter- 
national Labour Organization (ILO) (Louis 2016), a tripartite international 
organization established in 1919, in which the combination of methods has 
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proved useful in order to unveil the above-mentioned processes. 


(1) 


Highlighting change and continuity: Combining methods was 
rendered necessary by the specific temporality of my research 
question. Relying on multiple sources, especially archives and 
interviews, was necessary to prove the resilient character of my 
research question over time. I reconstituted the “chronology” of 
the controversies associated to representativeness. However, as 
archival records ended in the early 1990s, mainly for disclosure 
reasons, interviews were necessary to continue the investigation 
up to the 2010s. To put it more bluntly: interviews started where 
archives stopped. Based on archive analysis, I tested the resilient 
or, on the contrary, the conjectural character on debates around 
representativeness. Thanks to this combination, I was able to 
assert that the debate on representativeness within the ILO was 
a century-old debate (continuity) but that its politicization and 
intensity differed according to specific sociohistorical contexts 
(change). 


Challenging dominant discourses: One dominant view both out- 
side and inside the ILO is that workers’ and employers’ delegates 
are adversaries who are constantly fighting to attain a compromise 
that will eventually be endorsed by the governments. Such view 
is confirmed when we study the ILO only through archives like 
records of proceedings or minutes translating the (public) posi- 
tions of each group. However, observation and interviews allowed 
me to qualify this view and to shed light on more cooperative 
relationships between workers’ and employers’ representatives, 
but also friendships among a group of people who were sharing 
a specific institutional and personal experience due to their ILO 
membership. Archives, especially when they are institutional and 
official, most of the time remain silent on informal relationships 
and interpersonal conflicts as well as individual emotions and 
feelings. Moreover, as I was attending ILO meetings, I observed 
carefully mundane practices of members such as laughing, hug- 
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ging, and small talks that could imply more proximity and famil- 
iarity than archives and even interviews would ever reveal. I then 
used this observation to softly challenge my interviewees when 
addressing the kind of relationships that they had with delegates 
from other groups. I avoided mentioning the observation as 
such—which could have given the rather unpleasant impression 
that I had been “spying” on them—but rather made a general 
comment on the atmosphere: “I thought there would be much 
more tensions given the stake of the negotiations this year.” In 
other words, by concealing, even in an exaggerated way, our own 
perception of the situation, it can help the interviewees to expand 
more on their conception of the situation. 


(3) Resisting institutional amnesia: Part of my doctoral research 
focused on the negotiations of a sensitive and highly political con- 
stitutional amendment that aimed at suppressing permanent seats 
within the ILO Governing Body. I accumulated a considerable 
number of archives on it from the 1960s to the 1980s. At that time, 
this debate was not only considered legitimate, it also absorbed the 
energy of ILO members and staff. However, the examination of 
ILO proceedings from the 1990s seemed to indicate an extinction 
of the debate. When asking ILO members about it during inter- 
views between 2011 and 2013, I realized that most of them did 
not know about it or only in very vague terms. This was due to 
both turnover within delegations and the political will of dominant 
actors to not reactivate the debate. Scrutinizing the reactions of 
my interviewees when asked about this amendment, I was able to 
document feelings of embarrassment, ignorance, or indifference. 
Moreover, the fact that I constantly referred to a robust corpus of 
archives prevented interviewees from dismissing the question as 
irrelevant: they felt compelled to justify why they did not know 
about it or did mot want to act upon it. 


What Challenges? 


The main challenge in combining methods is to master each one in order 
for the research to (1) benefit from a genuine pluri-methodological input 
and (2) be recognized as valuable and relevant by your peers, including from 
other disciplines. Unfortunately, interdisciplinarity is more often a claim 
than a reality, especially at a time of increased specialization and sophis- 
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tication of methodological knowledge. For political scientists who do not 
have any background or training in history, going through archives might 
be a real challenge with entry costs (Garrigou 1989). Scholars, in particular 
students, should therefore be able to show, by carefully documenting their 
bibliography, that they are aware of the main epistemological debates exist- 
ing in each discipline and the methodological challenges associated with 
each technique and source. 

Beyond issues of recognition, it must be acknowledged that mastering 
one method already takes a lot of time, as proved by this handbook! In 
practice, not all methods can always be used and treated equally. Often one 
would feel frustrated for spending more time on one or two aspects of the 
research (in my case, interviews and archives clearly took most of my time 
at the detriment of more in-depth observation). However, beyond time, 
money, and institutional constraints, the most important aspect of com- 
bining methods is to make clear that they were all necessary to succeed in 
answering the research question and that it was not just a question of meth- 
odological curiosity. I would therefore advise researchers to keep in mind the 
following question: “What would I have missed had I not combined these 
three methods?” 


To Go Further 
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CHAPTER 22 


Computational Text Analysis 
and Archival Methods 


Evan Easton-Calabria and William Allen 


Researchers studying international organizations often rely on textual mate- 
rials produced by these organizations and held within archival settings, 
especially if their questions and periods of interest predate more modern 
materials and methods such as surveys or interviews (McGarr 2020). These 
materials, increasingly available in digital formats, potentially provide valu- 
able windows into organizational priorities and practices. While qualitative 
archival methods—often involving close reading—offer clear advantages of 
conceptual clarity and familiarity, they potentially incur high costs of time 
and human error. This chapter outlines how researchers can combine the 
power of simple computational textual analysis techniques with interpretive 
archival methods in ways that confer efficiency and empirical validity to 
studies of international organizations’ written outputs. 


What? 


‘The method we outline derives inspiration from the fields of computational 
and corpus linguistics, which seek to identify patterns of language use in 
sets of text (called “corpora”) that are systematically assembled for particu- 
lar purposes (Taylor and Marchi 2018). Our approach uses these patterns 
both as guides to potentially relevant portions of an archive and as forms 
of knowledge in their own right that can be used to address questions that 
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researchers ask about international organizations.' First, if researchers are 
already interested in a particular population, concept, or event (e.g., refu- 
gees, democracy), descriptive patterns can be identified of where those terms 
of interest tend to appear more or less frequently as a proportion of all the 
text in each document. Then more intensive close reading of those areas 
containing more mentions can be undertaken. By contrast, if researchers are 
interested in identifying terms that might characterize one portion of the 
archive (e.g., which issues appear more often in a certain time period), the 
frequencies of the words in one subset of documents can be compared to 
their frequencies in another subset. Having found those terms, researchers 
can scrutinize them further using qualitative methods. 


Why? 


History as a field, including the study of international organizations, is 
still coming to grips with recent leaps in digitization (Anderson 2007). On 
the one hand, the growing digital availability of sources offers the poten- 
tial to “democratize” archival research: researchers who previously could 
not travel to physical archives can now access many documents instantly. 
On the other hand, the growing volume and variety of materials presents 
large—sometimes prohibitive—costs of time and energy: closely reading 
every document is simply not practical (e.g., Quinn et al. 2010). An obvi- 
ous solution would be to select portions of an archive, dive into reading, and 
identify which documents (or portions thereof) seem “important” or “dif- 
ferent” from what one expects. However, source selection may be unknow- 
ingly yet systematically biased, such as in terms of what researchers consider 
to be interesting or relevant in the first place. Moreover, evaluating materials’ 
typicality is difficult without some view across the whole archive. This view 
could come from deep and long-standing engagement with an archive and 
the organization that produced it. Yet when initially exploring an archive, 
perhaps as part of a new project, researchers may not have this luxury. 

To address these challenges, social scientists increasingly turn to com- 
putational methods (Grimmer and Stewart 2013). On the one hand, these 


1. We make several key assumptions relating to research design. First, we assume that the 
assembled texts are justified as being (limited) representations of an organization. Second, we 
assume that the object of interest—such as an issue, actor, or process—has some observable 
manifestation within the corpora. Third, we assume that observable variation in the texts 
signals something important for our research question, whether that is a shift in attention or 
salience. For all of this, researchers must clearly justify their choices and theoretical stances. 
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approaches offer unrivalled strengths in efficiency and consistency. Unlike 
human coders, they can reliably evaluate many texts against many criteria 
without tiring. Meanwhile, by simplifying texts to their minimal features, 
they can identify patterns across the whole dataset that might not be readily 
apparent through sequential periods of reading, such as the positivity or neg- 
ativity of texts. However, these methods attract concerns about their validity, 
particularly when used in contexts where interpretation is key: statistically 
significant patterns may not be substantively significant, or even accurate at 
all (Brookes and McEnery 2018). There is growing consensus and evidence 
that these methods fall short of humans’ abilities when used uncritically and 
in isolation (van Atteveldt, van der Velden, and Boukes 2021). 

For some settings, having a consistent and clearly applied measure of 
features within texts may be sufficient, especially when that measure is vali- 
dated through coding done by humans who can more effectively adjudicate 
between clearly defined usages. However, archival researchers examining 
international organizations may have other kinds of questions in mind. For 
example, they may be interested in identifying beliefs and practices that 
are latent within texts (Bevir and Rhodes 2012). Yet landing upon these 
portions of digitized archives, and making sense of their wider significance, 
would be difficult without some sense of the whole set of materials. This is 
where our mixed-methods approach, drawing upon replicable techniques 
from computational and corpus linguistics, can help. 


How? 


We use three types of linguistic analysis arranged in different orders depend- 
ing on the research goal. Frequency analysis counts instances of a given word 
or group of words and then displays these frequencies either in terms of the 
whole corpus or differentiated by subsets of texts (“subcorpora.”) To enable 
comparison of frequencies arising from differently sized subcorpora, these 
frequencies are normalized into occurrences per thousand or million words 
(Taylor and Marchi 2018). Keyword analysis identifies terms that are central 
to a corpus or subcorpus by comparing it to another set of texts called a 
“reference corpus.” Words more unique to the corpus in question are better 
candidates for exploring what the texts might be about.’ Finally, concordance 
analysis displays the text around every instance of a word or phrase of inter- 


2. Different ways of calculating keyness potentially produce different candidate terms (see 
Gabrielatos 2018). 
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est, whether this interest is determined by frequency, keyness, or another 
measure. This provides a window onto how a given word or phrase is used 
in the documents, and a way to gain further qualitative access for either 
interpretation or disambiguation (for example, between asylum as used in 
either forced migration or mental health settings and environment to refer to 
political contexts rather than ecological terms). 

Our working example involves analyzing a corpus of ninety-three annual 
reports produced by the International Labour Organization (ILO) between 
1919 and 2015. These documents, made available online through the ILO’s 
digital archive, comprise more than eighteen million words.’ Our unit of 
analysis are individual reports, meaning that each year is represented by one 
document. While we did not divide the dataset into subcorpora (such as by 
decade), depending on the research question and types of archive materials, 
this is an option worth considering. 

‘The first step involves organizing and tagging the texts in preparation 
for analysis. This is a nonneutral process, involving choices about types of 
metadata to attach to the text, the most important units of analysis, and 
even which software or programming environments to use. We used the 
Sketch Engine (Kilgarriff et al. 2014), an online platform for lexicographic 
and linguistic analysis, but these techniques are not software-specific and 
can be implemented in different computing environments and packages. 
Regardless of the software or tools used to complete the quantitative analy- 
sis, we advise recording all the decisions made (and settings used) during 
the process, if one is not automatically produced. This not only supports 
reproducible and transparent research for future scholars but also jogs one’s 
memory upon returning to the project. 

From here, there are at least two possible pathways. Imagine our research 
were focused on refugees, and we were interested in ascertaining how salient 
refugees have been for the ILO throughout the twentieth century. The ILO 
is not usually publicly associated with forced migration issues involving 
refugees. Therefore if we were to rely on close reading of these documents 
to seek evidence of the ILO’s relationship with refugees, we would likely 
incur high search costs for uncertain results. Therefore we could measure 
the normalized frequency of the terms refugee and refugees (throughout this 
example, italicized words refer to the terms themselves rather than the group 
or population). In our research, doing so revealed three key periods dur- 


3. See https://labordoc.ilo.org/discovery/search?vid=41ILO_INST:41ILO_V2. A small 
number of reports were not machine-readable or published during this period. 
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ing which these terms were particularly visible as a proportion of all words 
in each annual report: 1921-1928, 1948-1952, and 1982-1984. Two of 
these periods—the interwar and post-WWII periods—fit with established 
histories of the ILO (Van Daele 2005). However, we also found evidence 
of a third moment of heightened refugee salience: in 1984, the ILO and 
UNHCR signed a memorandum of understanding (MoU). This opens fur- 
ther questions that archival research methods are particularly well-suited to 
answer, such as interpreting what these shifts in the ILO’s attention signified. 

Here we could use concordance analysis of the text surrounding instances 
of refugee(s) to focus attention on the portions of the documents explic- 
itly mentioning these terms. This stage of the analysis, supplemented with 
archival material beyond the corpus, involved more close reading and distin- 
guishes our approach from Mitrani and Noy’s chapter on computerized text 
analysis (chapter 17). This qualitative work suggests that the ILO became 
involved in refugee affairs when other humanitarian agencies including 
UNHCR emphasized the importance of refugee self-reliance, and advocated 
for livelihoods and employment as important ways to achieve it. These kinds 
of insights, when appropriately situated within theoretical expectations, can 
help archival researchers address interpretive questions about motivations 
and strategies. 

By contrast, imagine we were interested in identifying which issues were 
more salient for the ILO in some periods compared to others. Keyword 
analysis can generate lists of terms that may be good candidates for further 
qualitative exploration using concordance analysis as above. For example, 
have some issues typified more recent ILO bulletins compared to older ones? 
Addressing this question required drawing upon prior historical research 
that suggested three broad periods of international humanitarian focus on 
forced migration and livelihoods: 1919-1939, 1944-1979 and 1980-2015 
(Easton-Calabria 2015). Then, using 1919-1979 as the reference subcorpus, 
we compared the normalized frequencies of words appearing in the 1980— 
2015 bulletins against those appearing in the reference group. Words such 
as HIV, gender, and globalization were not only more frequent in recent 
ILO bulletins but also were entirely unique to this period: we did not find 
any mentions of these terms in the reference corpus. This type of analysis 
is important in its own right, particularly for addressing questions about 
how organizations’ agendas shift over time. Yet, in combination with con- 
cordance techniques, it also allows investigation of how terms appearing in 
both periods may have subtly different meanings. 
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What Challenges? 


Even as our mixed-method approach offers ways of effectively identify- 
ing, accessing, and interpreting archival observations, there are still several 
challenges. First, it assumes that researchers can access (or create) digitized 
documents of sufficiently high quality to be machine-readable. While his- 
torical documents are increasingly made available online, poor image qual- 
ity impedes automated textual processing. Similarly, primary handwritten 
documents are more difficult to process without human readers. Therefore 
our approach relies on the availability of typed and digitized corpora. For- 
tunately, developments in optical character recognition (OCR) are making 
this process easier and more reliable. 

Second, quantitative techniques—even relatively simple ones such as 
counting words—do not absolve researchers of either avoiding serious qualita- 
tive and interpretive work or accounting for their own (and possibly unno- 
ticed) biases in making those interpretations. Moreover, our approach needs to 
be embedded within strong designs and theoretical expectations as appropriate 
for researchers’ disciplinary norms.* Although computational techniques may 
provide signposts toward key parts of the archive, we advise not to underesti- 
mate the amount of time and energy needed to make sense of these patterns 
and show their wider significance for a given discipline or audience. 

Finally, the qualitative components of our approach still require the 
strengths and sensibilities of archival researchers. Reading “along,” “with,” 
and “against” the grains of archives (Stoler 2002) remains vital for both 
interpreting the quantitative patterns and providing the qualitative nuance 
necessary to substantiate them. 
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CHAPTER 23 


Qualitative Comparative Analysis 


Ryan Federo 


‘The growing interest in the configurational perspective to understand causal 
complexity has brought the spotlight on qualitative comparative analysis 
(QCA). However, the prospect of using QCA as a research technique to 
understand international organizations (IOs) just recently emerged in the 
literature (e.g., Federo and Saz-Carranza 2018; 2020; Binder 2017). This 
chapter sheds light on the nature and specificities of QCA in order to encour- 
age IO scholars to adopt the technique for their future research endeavors. 
For instance, some of the research questions that QCA can answer include 
understanding how multiple factors can combine to result in 1O-related 
outcomes and how IO characteristics complement or substitute each other 
in configurations to explain different outcomes. 


What? 


QCA is a research technique initially developed by Charles Ragin in 1987 
for subjects on comparative politics and historical sociology during the late 
1980s and early 1990s. Its initial purpose was to empirically examine a limited 
number of macrolevel phenomena that are relatively large for comparative 
case study (qualitative) and yet too small for statistical (quantitative) research 
designs. QCA aims to “integrate the best features of the case-oriented approach 
with the best features of variable-oriented approach” (Ragin 1987: 84). It is 
distinct from other research methods, as it draws on a configurational perspec- 
tive to view social phenomenon as a constellation of interconnected elements 
that should be analyzed holistically, rather than individually. 
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QCA underscores that causality is complex, characterized by three prin- 
ciples: 1) conjunction, referring to the notion that multiple, independent 
causal attributes jointly produce an outcome; 2) equifinality, suggesting 
that different combinations of conditions yield the same outcome; and 3) 
asymmetry, pertaining to the possibility that both presence and absence of 
attributes would be associated with the outcome. Since the predominant 
research tools to understand complex phenomena are not designed to cap- 
ture the three principles of causal complexity, QCA has now become the 
widely embraced research tool for empirically examining and disentangling 
causal complexity. In contrast to correlation-based techniques relying on 
linear relationships and examining net effects of individual factors, QCA 
explores the multidimensionality and interplay of factors to identify their 
joint effects. It uses set theory through Boolean algebra to investigate unions 
of set memberships by analyzing how the presence or absence of different 
conditions combine within a bundle to consistently influence an outcome 
of interest. 

More than half of academic journal publications that applied QCA are 
still in political science, followed by sociology, corresponding to Ragin’s dis- 
ciplinary domains. However, a growing number of social science fields such 
as management and public administration already have embarked toward 
adopting QCA-based studies. Although a late mover, the international rela- 
tions (IR) field also has gained ground in applying QCA. For instance, one 
of the earliest applications of QCA in IR is Drezner’s work in the sanctions 
literature (1999), which has continuously produced QCA-based studies 
(e.g., Boogaerts 2016). As Bennett and Elman anticipated (2007), QCA is 
becoming more frequent in IR studies such as the more recent ones by Arts 
and de Konings research on community forest management (2017) and 
Haesebrouck’s exploration of patterns regarding contributions to NATO's 
military campaign in Liberia (2017). Yet despite the evident interest on 
QCA in the IR field, a small number of studies so far have used the tech- 
nique to understand IOs (e.g., Binder 2017; Federo and Saz-Carranza 2018; 
2020; Heinkelmann-Wild and Jankauskas 2020). Thus I discuss here why 
QCA can be fruitful for IR scholars to study IOs. 


Why? 


The main advantage of QCA over other available research techniques rests 
on its ability to disentangle causal complexity. Such advantage is particularly 
important for understanding IOs that are nested within a complex system 
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comprising multilayered and multifaceted interdependent factors (Chau- 
doin, Milner, and Pang 2015). Three fundamental reasons support the rel- 
evance of QCA for future research on IOs. 

First, understanding IOs would benefit from configurational theorizing 
in QCA. Although not explicitly stated, IR scholars already assume a con- 
figurational approach by adopting a design perspective, in which IOs have 
organizational designs embodying multiple, yet interrelated, factors. How- 
ever, current theoretical and empirical investigations using a design perspec- 
tive focus on each explanatory factor’s net effect, while taking other factors 
constant. QCA enhances our understanding of IOs’ design by analyzing 
the cause-effect relationships of factors as a configuration. As such, we can 
identify whether IO attributes might be complements (i.e., enhancing each 
other’s effect) or substitutes (i-e., replacing each other) within a configura- 
tional design, thereby corresponding to the conjunction principle of causal 
complexity. Moreover, QCA can be used to perform multilevel analysis, 
which overcomes the challenge in studying IOs’ complex environment that 
involves multiple actors at different levels of analysis. 

Second, QCA helps in building better typologies. IOs vary along several 
design attributes in diverse settings; yet such differences might be related 
to the same outcomes, suggesting equifinality. Using QCA can systemati- 
cally group IOs into specific archetypes based on distinguishing conditions, 
rather than grouping them according to the commonalities among specific 
variables (i.e., cluster analysis). Although QCA does not allow researchers 
to identify which condition gives the greatest explanatory power, it could 
determine which attributes are core (i.e., definitive ingredients with stron- 
ger influence) or merely peripheral (i-e., with contributing influence) to the 
effects of configurations on the outcome of interest. Simultaneously, it could 
illustrate the possibility that the presence/absence of conditions may still 
produce the same outcome, coinciding with the asymmetry principle that 
cannot be captured by the symmetric nature of correlation-based methods 
(Fiss 2011). 

Third, QCA is flexible to analyze a wide spectrum of sample sizes, which 
is an important consideration for studying IOs. For instance, the IOs com- 
prising the UN system would be too small for statistical analysis and too 
large for comparative case study. The Correlates of War database shows 
around 350 active IOs; however, the amount of obtainable information var- 
ies among them. Thus whether information is available for a larger sample 
to conduct statistical analysis or information is only limited to a relatively 
smaller sample, QCA can be used for either research setting to perform a 
systematic comparison of the cases. 
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How? 


Although the process for conducting QCA-based studies is discussed heav- 
ily in the literature (see Ragin 2008 and Schneider and Wagemann 2012 for 
more details), this chapter provides a brief practical overview of the process 
(see fig. 11), following four steps: (1) model specification, (2) data collection, 
(3) data analysis, and (4) presentation of results (Parente and Federo 2019). 

Model specification. In applying a configurational logic, we need to 
establish strong theoretical and case knowledge about how different con- 
ditions mutually enhance their effects or replace each other when placed 
within a bundle. For instance, our research on IO boards relied on our 
knowledge of IO designs, as we identified only the relevant board features 
for our context. The key aspect is maintaining balance between the number 
of conditions and cases for the analysis to mitigate limited diversity, which 
occurs when configurations are not observed from the sample due to expo- 
nential increase in logically possible combinations when adding conditions 
to the model (Marx 2010). For example, we used five conditions to explain 
the outcome of interest, given the small sample sizes of fifteen and thirteen 
IOs for the analysis. 

Data collection. The next step involves selecting theoretically defined 
cases to ensure that the sample fits for the research. Similar to case-based 
research designs, case selection in QCA is purposeful; and in contrast to 
statistical research designs, random sampling is not advised, since outliers or 
deviant cases might be relevant to explain the relationship between the con- 
ditions and outcome (Greckhamer, Misangyi, and Fiss 2013). For example, 
even though researchers already have a set of cases, it is important to have 
a strong understanding of the cases to ensure that they are comparable and 
therefore QCA would fit as the technique for the analysis. 

Data analysis. Before analyzing the data, the conditions and outcome 
need to be transformed into set memberships, either as crisp or fuzzy sets 
(Ragin 2008), through calibration by using set-membership thresholds 
from theoretical/substantive grounds. However, calibration can also be done 
using thresholds based on the distribution of the dataset, particularly for 
exceptional instances where no theoretical basis exists in the literature. 

Once the data is calibrated, the next step is building a truth table show- 
ing the logically possible configurations with their corresponding number of 
observations. Thereafter, a necessity analysis should be performed to iden- 
tify whether a specific condition is always producing the outcome of inter- 
est. A sufficiency analysis follows by minimizing the truth table to ascertain 
whether conditions or combinations of conditions constantly produce the 
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outcome. Although the analyses can be done manually, it is recommended 
to minimize the truth table using a software such as fs/QCA, QCA packages 
for R or Stata, and Tosmana. In my own research, I primarily use fs/QCA 
because of its more user-friendly interface than other applications. Impor- 
tantly, a best practice is to perform necessity and sufficiency analyses for 
both presence and absence of the outcome to determine that configurations 
showing presence of the outcome are not subsets of configurations reflecting 
absence of the outcome, and vice versa. 

Sometimes, truth tables have unobserved rows, known as counterfactu- 
als. QCA allows researchers to account for counterfactuals in the analysis. 
If we want to include counterfactuals during the analysis, we could include 
both easy (i.e., consistent with theoretical knowledge and empirical evi- 
dence; for instance, the unobserved rows follow the theoretical expectations 
of having no cases that should exemplify the configuration) and difficult 
(i.e., consistent only with empirical evidence; for example, the unobserved 
rows are present because of the lack of cases showing the configuration) 
counterfactuals to produce parsimonious solutions or only the easy coun- 
terfactuals to produce intermediate solutions. Alternatively, we may choose 
not to include any counterfactuals during the analysis to produce complex 
solutions that represent findings as closest to the data as possible. 

Nevertheless, QCA is sensitive to several methodological decisions. 
Therefore to ensure validity of the findings, robustness or sensitivity checks 
are recommended to mitigate alternative conclusions. For instance, in our 
QCA papers on IOs, we performed model respecification by adding or 
dropping conditions during the analysis and modification of the calibration 
thresholds as robustness checks. 

Presentation of results. The configurations from the analysis can be pre- 
sented as a Boolean formula or a configuration table using the notation 
suggested by Ragin and Fiss. I prefer to use the latter to help the readers 
visualize the resulting configurations. 

Last, to interpret the findings, researchers should refer to the cases to 
make sense of the configurations, whether they are exemplified by typical or 
deviant cases. The cases also allow the researcher to identify any latent attri- 
butes underlying the resulting configurations. For instance, in our research 
on IGO boards’ monitoring, case knowledge was instrumental for identify- 
ing two dimensions that formed the basis of the typology. 
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What Challenges? 


Although QCA has several advantages to study IOs, we may also face chal- 
lenges in using the technique. First, the main challenge concerns whether 
QCA would be the appropriate approach for the research endeavor. To jus- 
tify the use of QCA, the research should involve configurational theorizing, 
the three principles of causal complexity, and case knowledge (Parente and 
Federo 2019). A common pitfall is mechanistically using QCA by merely 
mixing several conditions to observe how their combinations would be asso- 
ciated with an outcome. In studying IOs that are inherently characterized 
with a multitude of factors, how to reduce complexity into a reasonable 
amount of theoretically defined conditions to explain an outcome perhaps 
poses a challenge for researchers. 

Second, IOs are a different breed of organizations, which might be 
challenging to have a theoretical basis for data calibration when conduct- 
ing QCA. Although an alternative way is through a borrowing approach in 
which researchers benchmark thresholds by using those from other types 
of organizations, this might not always be applicable when studying IOs 
because of their unique characteristics and activities. For instance, opera- 
tionalizing organization size using number of employees and annual turn- 
over may not be the same in IOs as with firms and public organizations, 
since IOs sometimes have a working force that is not employed by the 
organization and these organizations rarely have revenues because they rely 
on member state funding. Moreover, in the absence of theoretical basis, it 
might not be recommended to use calibration thresholds based on data dis- 
tribution of smaller sample size studies, since such calibration is highly sensi- 
tive to the inclusion or exclusion of cases. Transparency in methodological 
decisions and strong case knowledge are key to support these decisions when 
using the technique. 

Finally, QCA is still at an embryonic stage. Although QCA is argued to 
be a long shot to supplant the conventional regression-based methods to 
conduct analysis in large-N sample studies and falls short in providing com- 
parable in-depth characterization of the cases offered by qualitative methods, 
it is not intended to replace current methods used in the literature; rather it 
should be viewed as a complement to other methods to provide alternative 
means of theorizing and empirical testing. Moreover, the strength of QCA 
lies in its ability to straddle between variable-based and case-based research 
methods by systematically comparing cases to identify patterns that could 
show which attributes would be necessary or sufficient to explain certain 
phenomena. Nevertheless, a configurational approach by theorizing and 
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empirically examining IOs using the logic of necessity and sufficiency offers 
fresh perspective and fertile ground to understand IOs’ complex world. 
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CHAPTER 24 


Structured, Focused Comparison 


Vytautas Jankauskas, Steffen Eckhard, and Jörn Ege 


Structured, focused comparison (SFC) allows a structured comparison of 
several cases (e.g., six international organizations), whereby the researcher 
conducts in-depth analysis within each case based on a standardized set of 
variables and general questions. The design not only increases the external 
validity of findings but also allows for cross-case comparison and a fine- 
grained theoretical analysis. Typical applications are research questions 
focused on processes or mechanisms and those that allow for the interplay 
of several interdependent conditions (causal complexity). 


What? 


The method of structured, focused comparison was first introduced in the 
early works of George (1979), George and McKeown (1985), and George 
and Bennett (2005). It aims at a systematic case comparison, which is built 
on an in-depth analysis within each case, while simultaneously allowing for 
the cumulation of findings across cases. The method can thus be positioned 
between small-N qualitative and large-N quantitative research. On the one 
hand, it rests on a process-based logic; on the other, it also relates to quan- 
titative research from which it “borrows the device of asking a set of stan- 
dardized, general questions of each case” (George 1979). This makes the 
method structured, since the same set of general questions—reflecting the 
research objective—guides data collection and is scrutinized in each case 
under investigation. The method is focused, as the researcher only deals with 
those variables of each case that are expected to be theoretically relevant. 


301 


302 International Organizations and Research Methods 


The following example illustrates the logic of SFC. In their study, Knill 
and colleagues (2019) defined and conceptualized administrative styles in 
IOs and explained their variation based on comparative case studies of four 
IOs. Selecting cases based on a most similar system design, they argued that 
the varying level of external challenges that IO secretariats face determines 
their administrative styles. For the empirical investigation, the authors used 
interview data and a survey of IO staff. Their analysis was thereby structured 
as the authors formulated general indicators that reflect both the dependent 
and independent variables and can be transferred to study further IOs. It was 
also focused as it only dealt with those theoretical features that were directly 
linked to administrative styles—IOs’ internal and external challenges and 
functional or positional orientations of IO bureaucracies. 

Historically, the SFC approach was motivated by the criticism of single- 
case studies, especially their noncumulative nature of empirical findings. The 
method was developed with the purpose of improving historical analysis of 
foreign policy events by accumulating findings from comparable case analy- 
sis into a broader theory. This should have discouraged “policy-makers from 
relying on a single historical analogy in dealing with a new case” (George 
and Bennett 2005: 67). 

Analogously, we suggest that IO scholars bring their attention back to 
SFC to combine qualitative and quantitative research methods to facilitate 
larger collaborations between IO scholars who are ready to (1) agree on the 
same theoretical frame and variables, (2) commit to finding similar data 
sources, and (3) trace processes with attention to similar details. IOs can 
be treated as complex problem-processing systems (Reinalda and Verbeek 
2004; Rittberger, Zangl, and Kruck 2012), so researchers who study them 
should be able to capture those interacting components that are relevant for 
specific research objectives. In this manner, SFC provides a systematic and 
rather simple way to increase studies’ external and internal validity. 


Why? 


The methodological divide between quantitative and qualitative research 
designs is a well-known phenomenon in political science. Research of inter- 
national organizations (IOs) also faces the challenge to balance between 
maintaining internal validity of findings and ensuring their generalization in 
other settings (see Eckhard and Ege 2016). Against this backdrop, the struc- 
tured, focused comparison offers a promising alternative for scholars seeking 
to explore the methodological middle ground. By combining features from 
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both qualitative and quantitative methods, we argue that SFC can increase 
validity, reliability, transparency, and scope of IO analysis, while at the same 
time bridging the traditional quantitative-qualitative divide. 

First, SFC offers a coherent framework, standardized data sources, and 
transparency over how conclusions are drawn systematically from the sources 
often lacking in single-case studies and small-N comparisons. Single-case 
research design is usually criticized due to its tendency “to go its way, reflect- 
ing [the] special interests of each investigator and often being unduly shaped 
by whatever historical data was readily available” (George and Bennett 2005: 
70). SFC, by contrast, offers a coherent, selectively focused treatment of 
empirical cases that enables better follow-up research by colleagues work- 
ing on other IOs or trying to understand over-time changes when studying 
organizations at later points in time. When it comes to controlled compari- 
sons, the combination of a theoretical focus and a structural case examina- 
tion allows for a systematic comparison across cases. If new cases of the same 
class of events emerge, they can be easily included into the existing SFC 
frameworks, which fosters the cumulation of findings over time. 

Second, SFC also benefits large-N analyses, since the results from the 
SFC applications could be used for case selection and further theory-testing 
endeavors. The prime methodology used to analyze large-N data covers the 
whole range of (usually regression-based) statistical models. More recently, 
new methodological approaches such as quantitative Social Network Analy- 
sis (SNA) and quantitative text analysis have gained ground in IO research. 
However, quantitative studies still rely on a few larger datasets that have 
not necessarily been designed to understand intra-IO dynamics (such as the 
Correlates of War project) or have known inconsistencies (such as the United 
Nations System Chief Executives Board for Coordination data). Where 
there is systematic data on organizational structures and dynamics, it is often 
limited to one or two organizations. Furthermore, quantitative studies tend 
to merely test and—sometimes—modify well-known theoretical assump- 
tions rather than bringing forward innovations in the theoretical debates of 
IO research. In light of this, SFC results could serve as a good starting point 
to establish plausible medium-N theories or to generate hypotheses that may 
be sustained with more large-N endeavors. 

Finally, the SFC method seems to be particularly applicable to some of 
the latest trends in IO research. The first trend points toward increasing 
interdisciplinarity in the study of IOs. For instance, while the interest of 
international relations (IR) in IOs remains high (at least after IR scholars 
have acknowledged IOs as autonomous actors; see Reinalda and Verbeek 
1998), IOs are being increasingly examined by other political science sub- 
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disciplines such as public administration, public policy, or comparative 
politics. In this regard, the use of general questions and focused theoretical 
frameworks (e.g., Ege, Bauer, and Wagner 2020) fosters the accumulation of 
knowledge across the increasingly diverse subfields of IO literature. 

The same holds for the second trend, which relates to the increasing 
digitalization and availability of data that can be used in the study of IOs. 
Arguably, the problem IO researchers are increasingly faced with is less a lack 
of information than the availability of too many (fragmented) data sources. 
In this light, the use of SFC would bring more methodological clarity and 
transparency. One of the key benefits of the SFC method is that scholars can 
refer to existing findings and build upon them by applying the same set of 
general questions and theoretical elements to further studies. This implies 
that both the formulation of data as well as the specification of concepts have 
to be clearly defined, otherwise the structural and focused conditions of SFC 
are not met. Considering the critique by Church and McCaffrey (2013) on 
IO documentation and available information (i.e., that IO research lacks 
systematic and comprehensive IO data collections), meeting the require- 
ments of SFC could help scholars increase the replicability of their data, thus 
fostering methodological clarity and transparency. 

Overall, the use of SFC might bring the advantage of combining an 
in-depth case-oriented perspective with a comparative, more systematic 
approach. SFC is best applicable to so-called causes of effects types of 
research questions where scholars look for the causes for the occurrence of an 
observed phenomenon instead of the effect of one isolated factor (Mahoney 
and Goertz 2006). Naturally, SFC cannot be expected to solve the well- 
known problems related to the observation and analysis of complex IO reali- 
ties. What it can do, however, is to outline a way in which a more systematic 
design of IO research can increase the generalizability of our conclusions 
and facilitate cooperation and knowledge accumulation in the future. 


How? 


This section highlights the main steps and presents some basic illustrations 
from our own research to show how the method could be applied to IO 
questions. In general, SFC case study data is defined and standardized by 
a set of general analytical categories, which are empirically linked through 
mechanistic processes. This allows a structured comparison of several cases, 
which is based on a careful case selection during which the researcher 
attempts to manipulate relevant explanatory variables (analogous to Mill’s 
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method of agreement or difference). SFC is therefore different from the 
congruence analysis that looks at whether the expected values of dependent 
and independent variables are observed empirically (pattern matching; see 
George and Bennett 2005: 181). 

Mote specifically, the following steps should be followed in designing a 
structured, focused comparison study (see George and McKeown 1985): 


e First, scholars should clearly spell out the research problem and 
objectives under investigation. For instance, in his study, Eckhard 
(2016) investigates what determinants enhance or impede organi- 
zational performance in postconflict police reform. 


e Second, researchers should specify the elements (variables, con- 
ditions, etc.) that will be used for the comparison. For instance, 
the main challenge in the study by Eckhard (2016) was to delimit 
explanatory factors linked to the organization from such factors 
linked to the country context in which missions acted. Inter- 
nally, organization and management theory provided a conceptual 
framework consisting of six categories of strategic management for 
comparison. Externally, international relations theory served to 
specify outside conditions as alternative explanations for mission 
performance. 


+ Third, case selection must follow a careful consideration of the over- 
all case universe and potential control variables (as well as alterna- 
tive explanations). In this line, Eckhard (2016) conducted four case 
studies that were structurally selected based on internal and exter- 
nal factors. Internally, Germany, the EU, and the OSCE operated 
along different managerial strategies. Externally, Afghanistan and 
Kosovo served to compare police reform in a more or less challeng- 
ing context. With the EU acting in both countries, a control case 
prevailed that allowed delimiting internal from external explana- 
tory factors. Ideally, the number of selected cases is large enough 
to allow for variation in both the key explanatory variable (e.g., 
internal management) and the context conditions with potential 
causal relevance (e.g., country context). 


e Fourth, the variance of variables or conditions should be described 
in detail—taking into consideration existing theory—and general 
analytical questions should be clearly formulated and reported. In 
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Eckhard (2016), the efforts of process tracing in each case were 
focused on by a common framework of six analytical categories 
of strategic management, which in turn allowed for cross-case 
comparison and the isolation of causally relevant variables. This 
way, SFC enabled the author to benefit from extensive interview 
data and rich qualitative insights about individual missions and 
causal mechanisms within each, while at the same time compar- 
ing mechanisms across cases to enhance the external validity of the 
explanation. In this example, it turned out that the EU’s internal 
management procedures (formalized planning in headquarters, 
lack of discretion at mission level, high frequency staff rotation) 
undermined the performance of assisting police reform not only in 
Afghanistan but also in the comparatively “simple” case of Kosovo. 


What Challenges? 


One of the main challenges of SFC is to formulate questions that are rel- 
evant for the specific case but at the same time can be applicable to the 
overall universe of cases. The same goes for its findings: the strength of SFC 
is internal validity regarding mechanisms and processes, but with only few 
cases there may be exogenous factors that the researcher did not control for. 
Case selection therefore is crucial. However, selecting the right cases may at 
times be challenging because the distribution of values in the population 
may be unknown or can only be identified in postanalysis. 

Another challenge is the integration of SFCs in the state of the art. Qual- 
itative studies are at times difficult to integrate with the state of the art, espe- 
cially regarding more quantitative papers, because they differ substantively 
in the operationalization of variables. Researchers should therefore be extra 
cautious to ensure the compatibility with the state of the art, for instance 
by discussing how other (more quantitative studies) operationalize a given 
phenomenon and how this relates to their own approach. 

A last, more practical challenge is about the space provided by publica- 
tion outlets. SFCs often require researchers to report on a lot of qualitative 
data. Whereas single-case studies might just fit within the scope of typi- 
cal journal article, reporting a comparative qualitative study with less than 
10,000 words is often difficult. This is why SFCs might be more appropriate 
for book-length publication formats, or more recent hybrids such as Cam- 
bridge Elements or Palgrave Pivot with 25,000 to 50,000 words. In fact, the 
recent emergence of these publication formats might also be a chance for 
more SFCs being implemented in the future. 
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CHAPTER 25 


Process Tracing 


Svenja Rauch! 


“Process was the policy” (Maurice Strong in Weiss et al. 2010: 303). As high- 
lighted by the secretary-general of the UN Conference on Environment and 
Development, the process of getting to an outcome at the 1992 Earth Sum- 
mit was as important as the outcome itself. Processes within international 
organizations (IO) at large, including negotiations and drafting of reports, 
provide critical insights into the inner workings of multilateralism. Process 
tracing allows researchers to closely follow and qualitatively examine com- 
plex multilateral processes as they unfold, documenting critical moments 
and evaluating the mechanisms that led to a specific outcome. Used for 
conducting within-case analysis, rigorous process tracing requires the gen- 
eration and triangulation of a significant amount of data through a combi- 
nation of research methods, including but not limited to discourse analysis 
(see chapter 11—Discourse Analysis), direct or participant observation (see 
chapter 1—Direct Observation and chapter 2—Participant Observation), and 
interviews (see part 2—Jnterviewing). To illustrate the application of a pro- 
cess tracing approach, this chapter draws on my PhD research on the multi- 
lateral processes that led to the adoption of the 2030 Agenda for Sustainable 
Development (UN General Assembly 2015). 


1. The views expressed herein are those of the author and do not necessarily reflect the 
views of the United Nations. 
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What? 


Having first emerged in the late 1960s in US scholars’ work on cognitive 
psychology, George (1979) introduced the term “process tracing” in politi- 
cal science. It was, however, not until the early 2000s that process tracing 
gained prominence in the social sciences. While not explicitly describing his 
approach as process tracing, Schimmelfennig in his work on enlargement of 
the European Union (EU) and of the North Atlantic Treaty Organization 
(NATO), for instance, analyzed “the causal process that links independent 
and dependent variables” (Schimmelfennig 2003: 13). 

George and Bennett first defined process tracing as examining “histories, 
archival documents, interview transcripts, and other sources to see whether 
the causal process a theory hypothesizes or implies in a case is in fact evi- 
dent in the sequence and values of the intervening variables in that case” 
(2005: 6). Process tracing thus constitutes an approach to account for a 
causal mechanism, “the pathway or process by which an effect is produced 
or a purpose is accomplished” (Gerring 2007: 178, quoted in Bennett and 
Checkel 2015: 76). 

While controversies around the notion of causal mechanisms persist, 
process tracing has demonstrated its value as a tool for qualitative analy- 
sis. Examining a series of political or social events, process tracing seeks to 
reveal the intermediate steps and the temporal order in which they occur, 
as well as the mechanisms that led to a specific result. This approach proves 
particularly valuable in the context of multilateral processes in which actors 
defend a multiplicity of interests. Process tracing allows researchers to reveal 
the factors that make those interests gradually converge in an outcome 
that is acceptable to all. To establish a sequence of events, a process tracing 
approach combines deductive and inductive elements (Bennett and Checkel 


2015: 17). 


Why? 


Process tracing explores mechanisms within IOs to describe both institu- 
tional structures and practices, as well as the role of individuals in their 
daily work. Increased attention to process and mechanisms has allowed 
“capturing better the complex social reality” (Bennet and Checkel 2015: 
89) of IOs, and their influence on state behavior and interests. Schimmelf- 
ennig, for instance, studied why the EU and NATO decided to expand to 
the east, whereas Autesserre (2009) examines the reasons for international 


310 International Organizations and Research Methods 


peacebuilding failures. In both cases, process tracing helped reveal the moti- 
vation of actors, their constraints and the sequencing of actions that shaped 
decision-making processes. 

While process tracing can be applied to study a wide range of phenom- 
ena, this chapter seeks to illustrate the approach by focusing on international 
negotiations. Process tracing proves valuable in identifying the negotiation 
trajectories (Bourque and Thuderoz 2011: 115) and the multiple interactions 
that steer the latter by going beyond the focus on their outcome. Thus process 
tracing may shed light on three aspects of the notion of “process” (Bourque 
and Thuderoz 2011: 110-11): the sequence of events or different stages of 
the negotiations; the dialectic process of formulating offers and counter- 
offers as IOs seek to facilitate joint decision-making by their membership 
(Zartman and Rubin 2000: 12); and the set of tactics used by negotiators. 

A descriptive reconstruction of a process constitutes a first step, followed 
by an analysis of whether causal inferences can be drawn based on the empir- 
ical evidence collected. Identifying the different stages of the negotiations, 
the causal chain (George and Bennett 2005: 206) between a number of 
elements can be examined: (i) the agency of actors; (ii) the strategies they 
use to make the process manageable; (iii) the way actors frame their posi- 
tions; (iv) the coalitions they form; and (v) the trade-offs being ultimately 
achieved. However, negotiation processes are not linear: phases may overlap 
and turning points might occur serendipitously. Process tracing can account 
for a change in tactics of key players at different stages of the process. More- 
over, contextual elements, including the negotiation venue, the negotiation 
culture of the IO, and the political context in which negotiators interact, as 
well as the linkages of the deliberations to other multilateral processes, are 
important to understand the “give and take” in international negotiations. 
“Process dynamics” (Bennett and Elman 2006: 259) are salient to explain 
the various “ins and outs” of specific text proposals and help decipher the 
intricate language of international agreements. Furthermore, process tracing 
is dynamic in that it allows the researcher to account for surprises and adapt 
hypotheses based on the data generated. 

In combining various methods and anticipating alternative explanations, 
process tracing may offer a better understanding of why and how specific 
norms and policy narratives were created. The evidence collected may, for 
example, suggest a constructivist reading of the role of “norm entrepre- 
neurs” (Finnemore and Sikkink 1998) and ideas in international processes 
(see box r—Studying Ideas). At the same time, examining the same evidence 
through a realist lens may reveal underlying power dynamics. Process trac- 
ing thus explains not only the what, the outcome of negotiations, but more 
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importantly the how and why, the negotiation trajectory and the factors that 
shaped the latter, as well as the who, the agency of individuals. 


How? 


Researchers pursuing a process tracing approach “apply information- 
gathering techniques that have been developed by historians and ethnog- 
raphers, and they think very much like detectives and less like statisticians” 
(Blatter and Haverland 2012: 105). Process tracing, therefore, requires a 
significant amount of evidence to provide a comprehensive storyline, i.e., 
describing the structural conditions and the main phases of the process, to 
identify smoking-gun observations offering a detailed account of a given situa- 
tion, and to reveal confessions. The latter encompass statements unveiling the 
perceptions and underlying motivations of actors (Blatter and Haverland 
2012: 117) and can be analyzed to track how arguments are being used for 
shaping policy (Risse 2000). 

Aiming at a detailed description, preparatory work is fundamental to con- 
textualize the process at hand and to elaborate hypotheses on the expected 
negotiation behavior of key actors and interest coalitions. The compilation 
of a corpus of primary source materials and grey literature is a critical first 
step. These materials include archive material of previous international pro- 
cesses and frameworks, resolutions, decisions of intergovernmental bodies, 
and reports issued by the IO, among others. 

In the case of the negotiations on the 2030 Agenda for Sustainable 
Development, I analyzed previous agreements in the sustainable develop- 
ment field and written materials pertaining to the development workstream 
within the UN. Furthermore, knowledge of the rules of procedure is essen- 
tial to examine how the latter are being interpreted and applied, both by the 
cofacilitators of the process and the parties to the negotiation. Negotiators 
may also try to link the deliberations to other ongoing workstreams within 
the organization. Researchers should thus be aware of the “bigger picture” in 
which negotiations take place. 

To identify the overall narrative that underpins the 2030 Agenda, I used 
discourse analysis (see chapter 11—Discourse Analysis) to study statements 
issued by the parties to the negotiations, the various iterations of the negotia- 
tion text, and accompanying documentation provided by UN system entities, 
academia, and major groups and other stakeholders.” Discourse analysis (Wid- 


2. https://sustainabledevelopment.un.org/mgos (accessed March 31, 2021). 
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dowson 2007), on the one hand, helps discover “how meaning is negotiated 
between the members of a discourse community” (Angermuller, Maingue- 
neau, and Wodak 2014: 3). On the other hand, it is critical to understand 
the technical language of resolutions, negotiation documents, and reports to 
reveal the delicate political balance struck in the respective texts. 

Second, researchers need to closely follow the process of interest as it 
unfolds. While the analysis of written material provides a solid basis for 
tracing changes from one iteration of the negotiation document to the next, 
data collected through discourse analysis tells little about the setting in 
which the deliberations take place. Direct observation, therefore, can gener- 
ate additional data sets (see chapter 1—Direct Observation). It may yield 
many benefits, such as verifying the assumptions made on the behavior of 
different actors and revealing their day-to-day interactions within the IO 
and its “negotiation culture.” However, direct observation requires access 
to the fora in which negotiations take place, which may prove difficult and 
oftentimes implies financial resources. If it is not possible to follow the pro- 
cess in its entirety, an observation period needs to be chosen carefully. 

Direct observation can further prepare the ground for interviews to gen- 
erate additional qualitative data. Interview-based research helps elicit the 
views of individuals involved in the process in a limited timeframe (see 
chapter 5—Semistructured Interviews). The semistructured format offers 
“sufficient flexibility to accommodate the individual circumstances of each 
interview, while also providing a basic structure of questions that allows for 
comparison” (Lodge 2013: 187). With a set of preestablished questions, 
researchers may then tailor them to each interviewee. As interviews with 
international elites are—most of the time—“one-shot games” (Lodge 2013: 
194) and involve power asymmetries, each interview needs to be prepared 
thoroughly. Prosopography (see chapter 26—Prosopography) can be a useful 
tool in this regard.’ 

Through synthesizing and systematically triangulating the empirical evi- 
dence, a comprehensive storyline can be established that allows researchers 
to identify critical turning points of the process and serves as the foundation 
for analysis on whether causal inferences can be drawn. In their analysis, 
researchers need to be clear about the selection of data presented while, at 
the same time, critically examine alternative explanations for all relevant 
aspects of the outcome. Finally, researchers evaluate whether the findings 
can be generalized, linking back to IO theories. 


3. See, for example, JO BIO— Biographical Dictionary of Secretaries-General of International 
Organizations project, www.ru.nl/fm/iobio 
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What Challenges? 


Researchers need to be aware of the following both theoretical and practical 
challenges: 

First, debates on the approach, particularly on causal mechanisms, have 
implications for researchers’ choice between “process tracing,” “process- 
tracing,” and “causal-process tracing” (Blatter and Haverland 2012: 142). 
Researchers should thus be consistent in the terminology that they adhere to 
and be prepared to explain the specificities that this choice implies. 

Second, process tracing requires the generation of a significant amount of 
data. While gathering as much information as possible is key, data collection 
needs to be systematic and rigorous. Researchers are thus confronted with 
the question of choosing the starting and end points of data collection as 
well as the combination of methods. If data generation through direct obser- 
vation and interviews is envisaged, access to the negotiations and its various 
actors is as critical as is the timing: if interviews are scheduled too early in 
the process, the interviewer may not be able to tease out critical turning 
points. If interviews are conducted late in the process, negotiators might not 
be available due to unpredictable schedules and increasing politicization of 
discussions. Attending international meetings also involves resource require- 
ments both in terms of time and costs. 

Third, researchers need to carefully triangulate the data collected to 
increase the credibility of the explanation provided. This exercise involves 
anticipating alternative explanations from the outset and thoroughly evalu- 
ating them as the process unfolds, including being open for surprises. Coun- 
terfactual information can further be used to better grasp alternative paths 
of action. 

Fourth, synthesizing and presenting the considerable amount of evidence 
to readers can be challenging. Researchers need to find a balance between 
abstract, technical language and a narrative that brings the process to life 
and, at the same time, links process tracing back to IO theories. 

Fifth, process tracing is case-specific, and hence the causal mechanisms 
that it reveals may be difficult to generalize. While this limitation needs to be 
acknowledged, researchers can attempt to mitigate this challenge by carrying 
out an additional comparison between cases. 


To Go Further 
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CHAPTER 26 


Prosopography 


Aykiz Dogan and Frédéric Lebaron 


Analyzing individual trajectories through biographical material to study 
their social spaces can be particularly useful in the research on IOs. This 
“social space” approach initiated by Bourdieu and Saint Martin (1978) and 
developed by others (including Lebaron 2000; 2008; 2010; 2016; Charle 
1998; Sapirol999) uses the prosopographical method to study social fields. 
While it is most often employed to study elites, it applies perfectly to dif- 
ferent groups in an IO hierarchy, including decision-makers, staff members, 
employees, or national intermediaries. 


What? 


From Greek origin, prosépon signifies the characteristics of a person, whereas 
graphia means description (Verboven et al. 2007). As a method, prosopogra- 
phy implies “the investigation of the common background characteristics of a 
group of actors in history by means of a collective study of their lives” (Stone 
1971: 46). While there are various approaches to prosopography, it is first 
developed and mostly used as a historical research method especially since the 
1960s (for literature reviews see Charle 2001 and Keats-Rohan 2007). 

In our research, we use it not as an end but as a means to provide answers 
to research questions regarding large quantities of social actors to study the 
structuration of their social space and to investigate the relations between 
their positions, dispositions, and position-takings. Verboven et al. (2007) 
describe this kind of approach as “sociographic,” as it allows researchers to 
study the composition of a specific social group and its characteristic features 
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in the society based on a particular sociological paradigm that is, in our case, 
derived from Bourdieu’ relational sociology. “It is never, therefore, ‘neutral’ 
or merely descriptive” (Verboven et al. 2007: 38). This approach has been 
defined by Broady (2002) and Deene (in Keats-Rohan 2007) among others 
as “French prosopography” or “Bourdieusian prosopography.” 


Why? 


The prosopographical method allows researchers to study a group of social 
actors’ biographical elements in a comparative perspective (i.e., members of 
an organization, a social movement, an occupational group, an intellectual, 
political, economic or artistic circle, etc.). It is typically used to understand 
the social functioning of institutions, decision-making processes, and other 
political action as well as social structure, stratification, and mobility (Stone 
1971; Verboven et al. 2007). It allows researchers to objectify social relations 
as it aims “to collect data on phenomena that transcend individual lives” 
(Verboven et al. 2007: 41). Relations are understood here as shared socio- 
logical characteristics and differences, and not only personal or institutional 
connections. 

Our research shows that individual biographies matter especially when 
the analysis is comparative and pays attention to the multidimensionality 
of trajectories. The comparative analysis of indicators such as sociodemo- 
graphic, educational, and professional background; national and social ori- 
gins; and political, ideological, and intellectual orientations allows research- 
ers to show how biographical characteristics play a role in position-takings, 
hence influencing institutional decisions and policies. When applied in a 
diachronic perspective, examining different periods, it provides insights 
regarding the mechanisms of transformation and reproduction of the field. 
Furthermore, a prosopographical inquiry can help identify both normative 
and deviant trajectories and to build typologies (Lebaron and Dogan 2016). 

For instance, our ongoing prosopographical study on the governing 
actors of the financial monetary IOs, the Bank for International Settle- 
ments (BIS) and the International Monetary Fund (IMF), indicates that 
not only international but national affiliations are decisive in the access 
to top decision-making positions. It suggests a strong bias in favor of the 
Global North, a tendency that comparative research might reveal in other 
IOs as well. Furthermore, our studies on another international-regional 
organization, the European Central Bank (ECB), indicate many shared 
properties, especially regarding masculine domination, academic qualifica- 
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tions in economics, and professional careers in financial institutions, that 
structure the social space of financial-monetary IOs leaders (i.e., Lebaron 
and Dogan 2018). 

A properly configured prosopographic database also enables geomet- 
ric data analysis (GDA) for the visualization of connections and relations 
between individuals, the multiplicity of configurations and specific dimen- 
sions that structure a field (i.e., Lebaron 2000; 2008; 2010; Lebaron and 
Dogan 2016; 2018). It can also be configured to apply other quantitative 
methods such as network analysis to study network structures, personal and 
institutional relations, social capital of actors; or sequence analysis if data is 
available on career or institutional trajectories. 

This methodological perspective therefore might be very useful to study 
IO actors as a social space, connected to a set of national and supranational 
other spaces. There is indeed an increasing number of prosopographical 
studies contributing to the IO literature, especially on past (Tollardo 2014; 
Torsten 2019) and present IOs (i.e., Georgakakis 2010; Mourlon-Druol and 
Romero 2014) and international professional groups (i.e., Belém Lopes and 
Oliveira 2017). 


How? 


Prosopography involves the construction of a database that allows statistical 
analysis among others. We explain here these steps in detail. 


Study Design 


As in all scientific work, research questions and hypotheses guide the choice 
and elaboration of research tools. We identify the study population and pre- 
pare a prosopographic “questionnaire” as in a sociological survey. A prelimi- 
nary investigation is useful for defining the population and primary sources, 
and for building a first version of the questionnaire that will be adjusted 
during the research as we learn more about the population. 


Sources 


A prosopographical study mobilizes biographical material including archive 
documents and in-depth interviews. In our research, we combine published 
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sources, especially the Whos Who collections, with much less conventional 
sources through detailed research on the web, including specialized websites, 
online press, or LinkedIn. We strongly recommend noting information 
with their sources in a more detailed manner than is required for a survey 
research. Observing which kind of information is available for which indi- 
vidual from which sources contributes both to the study design and analy- 
sis. We recommend also taking screenshots or recording online sources that 
might change content or disappear from the web. Note-taking programs are 
helpful to construct descriptive records for individuals organized by tags. 
‘This step allows researchers to link individual and institutional records, to 
turn back to the source to read the full description when needed, to make 
double-checks and comparisons between sources, and hence to verify the 


credibility of data. 

Data Collection 

Our questionnaire usually starts with questions that specify the actor’s 
role in the institution that we study, then focuses on other characteristics 
depending on the population and research questions. The table below pro- 


vides some examples based on our research. 


Table 7. Example of Categories for a Prosopographic Questionnaire 


Role in the institution Ex.: Governor 

Time frame 2000-2010 

Sociodemographic characteristics Sex, date, and place of birth, etc. 

Education and academic qualifications Discipline, academic qualifications, univer- 
sity, etc. 

Professional background Employment trajectories, main career or 


work experience in different sectors, etc. 


National and international affiliations Board memberships, think-tanks, founda- 
tions, groups and networks, political party, 
etc. 

Political, ideological, and theoretical Religion, intellectual or political engage- 

orientations ments, public statements and positions, etc. 


Social origin Ex.: parents’ occupation 
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Data Entry 


The prosopographic questionnaire constructs a database with questions and 
their answers listed for each individual. Data entry should not be mixed 
with coding even when data is entered in a more or less harmonized manner. 
As a general rule, we suggest first entering detailed data before coding it in 
separate sheets. 


Coding 


Each question is in fact a variable and different answers might be regrouped 
into standardized modalities. We recode data in a more homogeneous way 
multiple times (especially when treating the data) reflecting on what kind 
of information is lost. This “cleaning” process adapts to the needs of the 
selected analysis techniques. 


Analysis 


‘The analysis starts during data collection and continues through coding, by 
noting preliminary observations and avenues for further inquiry. A proso- 
pographical database allows quantitative analysis, while it is also possible to 
provide answers to research questions without using statistical tools. In gen- 
eral, quantifying the results has many benefits such as objectifying descrip- 
tions by numbers that can serve as proof for arguments. In our research, we 
start by an exploratory data analysis before applying more complex methods. 
Frequency or pivot tables examined together with charts allow researchers to 
statistically observe the population and the general properties of their social 
space. Qualitative methods such as biographical case studies focusing on the 
most representative or marginal individuals are also very useful. The origi- 
nality of our own contribution is to combine the study of biographies with 
the GDA visualization techniques as well as other statistical tools, such as 
exploratory data analysis, linear regression techniques, or sociological (lexi- 
cometric) discourse analysis. 
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Visualization 


The analysis might involve visualization techniques. To learn more about 
mapping out the relations between individuals in terms of shared character- 
istics and divergences see chapter 18—Multiple Correspondence Analysis, and 
for visualizing their relations in terms of network connections see chapter 
19—Social Network Analysis. Discourse and content analysis methods might 
also be useful in visualizing the position-takings of actors and their ideologi- 
cal tendencies (see for instance chapter 17— Computerized Text Analysis). In 
any case, the questionnaire should be designed at the beginning according to 
the analysis method(s) chosen to answer the research questions. 


What Challenges? 


One of the most evident challenges in a prosopographical study is the 
unavailable information for certain questions and individuals. As in histori- 
cal research in general, it is important to acknowledge and precise the limits 
of the sources. The nonavailable information should be coded so. In fact 
it does not constitute a definitive problem, but on the contrary an object 
of analysis in itself assuming that data is not missing randomly. Observing 
which type of information is easily accessible and which is not provides valu- 
able insights regarding the study population and their social space. 

There are advanced methods to treat missing data depending on the 
selected statistical techniques. For descriptive statistics, we handle it in three 
simple ways. If many questions remain unanswered for an individual, we 
omit them from the population. If the responses are unavailable for a certain 
question, we note and exclude it (as a variable) from the analysis. Otherwise, 
we precise the number of available and nonavailable responses for each anal- 
ysis (hence forming an empirical sample for each question), acknowledging 
that the results will be biased and trying to identify in which regards. 

Information overload might also be troublesome both in terms of data 
collection and coding and analysis. For instance, in employment sequences 
we observe various individuals being a member of multiple company boards 
while serving as an administrator at an inter/national institution or lectur- 
ing at a university, or sometimes all and more at the same time. In fact, it 
indicates the multidimensionality of individual profiles cumulating multiple 
positions in various fields. As much as it is valuable for the analysis, it also 
poses certain complications. In our research, for instance, this kind of infor- 
mation is often unavailable at first sight, because unless a detailed CV is in 
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circulation, most of the biographies privilege one of the professional activi- 
ties at a time. We can hypothesize that either the individuals reporting their 
short biographies or the institutions publishing them make such choices. 
Hence observations on these institutional, public, and self-representations 
enrich the analysis. A second problem is the question of coding to efficiently 
integrate data into the statistical analysis as these tools usually function by 
regrouping singular categories (“Admin/Univ” constitutes a different cat- 
egory than “Admin” or “Univ”). A possible strategy is to count both for 
each category and compound categories. Third, not always the same kind of 
information is available in the same way for all. For instance, institutional 
roles might be detailed for certain individuals but very limited for others. 
We suggest focusing on samples or cases to make value of specific informa- 
tion, which otherwise limit a comparative perspective. 

Another challenge particular to the study of IOs and transnational actors 
concerns the reconciliation between “international” and “national” trajec- 
tories that proves to be complex even in a highly globalized world where 
common standards and norms structure national organizations. Hence 
data coding involves a continuous effort to reconcile national, regional, and 
international categories without ignoring or assimilating key specificities. 
Indeed, a privileged research objective should be identifying them. 

In short, even though there seems to be no strong specificity in studying 
the IOs with the prosopographic method, there are particular difficulties 
(Lebaron 2016). Nevertheless, when analyzed reflexively, these challenges 
might constitute valuable observations for the study. 


To Go Further 
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BOX W 


Research with LinkedIn 


Monique J. Beerli 


Founded in 2002, LinkedIn is a social networking service dedicated to 
“connecting the world’s professionals” with a reported worldwide member- 
ship of 690 million users (LinkedIn 2020). Designed to facilitate commu- 
nication between jobseekers and recruiters as well as among professionals, 
LinkedIn has the potential to be harnessed as a tool for social scientists in 
quests to access interviewees, collect biographic data, and analyze profes- 
sional networks. 

International organization (IO) studies scholars working with interviews 
typically offer snowball sampling, infiltration as a participant-observer, and 
allying with gatekeepers as techniques for securing dialogical opportunities. 
However, these strategies assume that the preconditions of access, such as 
knowing who to talk to and where/how they can be reached, have been met. 
As IO organizational charts and websites offer scant information on who 
does what, identifying interviewees and how to contact them might turn 
out to be surprisingly onerous—especially when dealing with geographi- 
cally and/or institutionally fragmented global professionals who cannot 
necessarily be located in a tangible space at any given moment. Render- 
ing individuals searchable by name, job title, organization, and geographical 
location, LinkedIn can be exploited as a global directory, albeit limited to 
registered users, to overcome germinal access obstacles. Potential interview 
subjects can then be contacted via LinkedIn’s messaging function or, when 
provided, by email. Don't be fooled though, messaging isn’t entirely free: for 
basic account holders, messaging is limited to individuals in your network, 
whereas paying subscribers can message who they wish. 

LinkedIn is also a treasure trove for researchers employing qualitative 
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as well as quantitative methods to investigate global dynamics of power, 
influence, and authority by mapping the structure of career pathways, pro- 
fessional networks, and social fields (Henriksen and Seabrooke 2017: 59; 
Kipping, Bithlmann, and David 2019). Resembling a CV though varying in 
their degree of detail (Zhu, Zhao, and Liu 2020: 91), LinkedIn profiles serve 
as keys to historicizing the emergence of new elites and professional groups, 
mapping career trajectories, or analyzing the types of resources (i.e., diplo- 
mas, organizational loyalty, linguistic capacities, etc.) required to accede 
positions of power. Displaying connections between users and user adhesion 
to groups, LinkedIn can equally provide the empirical base for virtual social 
network analysis (Papacharissi 2009). 

Eliminating certain hurdles, doing research online through tools like 
LinkedIn also comes with a range of concerns and responsibilities that merit 
consideration, from the validity and robustness of profile data to ethical 
dilemmas that arise when considering principles such as informed consent 
and anonymity. 
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CHAPTER 27 


Practice Analysis 


Vincent Pouliot 


In many ways, what structures the work and politics of international organiza- 
tions (IOs) are socially recognizable patterns of action, such as voting, budget- 
ing, staffing, peacekeeping, negotiating, reporting, monitoring, and so on. Of 
course, these prevalent practices owe in part to the rules that constitute the 
organizations, such as founding charters and formal procedures. But the pat- 
terns of action that characterize IOs are also largely informal, emerging from 
day-to-day activities and fast-changing political expediencies. Practice analysis 
targets the daily routines, bottom-up experiments, and organically emerging 
ways of doing things that give movement and life to IOs. 


What? 


Practices are socially meaningful and organized patterns of activities; in lay 
parlance, they are ways of doing things (Adler and Pouliot 2011). One can 
think of myriad practices, from handshaking in our everyday lives to diplo- 
macy in world politics. In a nutshell, anything that people do in a contextu- 
ally typical and minimally recognizable way counts as a practice. Practice 
tracing, then, aims at inductively and interpretively recovering the patterned 
ways of doing things that structure a social site of interest (Pouliot 2014). In 
the case of IOs, some practices are common to several of them (e.g., “hold- 
ing a bracket” as part of multilateral negotiations), while others are peculiar 
to each (e.g., the NATO practice of “joining the consensus”). Depending 
on the research question, practices may be construed at variable levels of 
aggregation. 
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At the analytical level, the concept of practice presents a number of 
opportunities, one of which being that practices are both particular (as con- 
textually embedded actions) and general (as patterns of actions). In other 
words, as a unit of analysis practices offer a common ground where concerns 
for contextual specificity and analytical generality can equally be met. In 
that sense, practice tracing is a hybrid methodological form that rests on 
two relatively simple tenets: social causality is to be established locally, but 
with an eye to producing analytically general insights. ‘The first tenet, drawn 
primarily from interpretivism, posits the singularity of causal accounts: it 
is meaningful contexts that give practices their social effectiveness and gen- 
erative power in and on the world. The second tenet, in tune with process 
analytics, holds that no social relationships and practices are so unique as to 
foreclose the possibility of theorization and categorization. Practice tracing 
seeks to occupy a methodological middle ground where patterns of mean- 
ingful action may be abstracted away from local contexts in the form of 
social mechanisms that can then travel across cases. 


Why? 


Practice tracing takes the reader on a journey to the engine rooms of world 
politics. By going inside IOs and looking at their everyday functioning, the 
methodology opens a window on a social theater whose inner dynamics 
have long been neglected. Analytically, the key payoff is to reconstruct inter- 
actions and situations in all their social productivity. Put differently, the 
everyday and the mundane are much more than a string of anecdotes: they 
are socially generative of broader political effects, for instance, in fostering 
inequality or in explaining the particular shape of a given outcome of inter- 
est. From the “infinitely small” emerge political orders. Tracing the ordinary 
operation of international pecking orders helps identify the concrete ways 
through which multilateral diplomacy produces inequality (Pouliot 2016) 
while focusing on mundane practices and symbols sheds light on how the 
European Union constructs its authority on a daily basis (McNamara 2015). 

Practice tracing is also particularly apt at capturing informality—once 
again a dimension of world politics that is often neglected in IR scholar- 
ship, perhaps because of the inherent empirical difficulties involved. Writ- 
ten rules and textual accounts are by nature incomplete and the study of 
practices invites a focus on whatever happens “between the lines” of IO 
activities. Several of these activities are unwritten, sometimes even hidden 
from sight, but that is no reason to discount them as insignificant at the 


Practice Analysis 327 


analytical level. Autesserre studies “on-the-ground peacebuilding” and iden- 
tifies international interveners’ dominant practices, habits, and narratives to 
better understand the ineffectiveness of peacebuilding (2014). In addition, 
the methodology of practice tracing steps away from the limitations of levels 
of analysis and preset categories of actors. By focusing on patterns of action, 
it does not matter where these activities take place or who they involve. By 
following practices regardless of who performs them, the framework admits 
much more variation in participants and sites than typical approaches would 
(Pouliot and Thérien 2023). Focusing on temporary international civil ser- 
vants, Bode shows, for instance, how these overlooked actors can act as 
agents of change within the UN (2015). 

It seems useful to conceptualize four analytically distinct spaces of the 
social: the situational (that is, the enacted practices within a given social 
locale), the dispositional (the historically accumulated know-how that goes 
into practice), the relational (the morphology of social ties), and the posi- 
tional (the location in a social configuration from which the practice is being 
enacted) (Pouliot 2016). Because they describe fairly different processes at 
the ontological level, situations, dispositions, relations, and positions cannot 
be grasped through the same analytical tools. A well-organized combination 
of various methods is necessary. 


How? 


At the most basic level, practice theory contains a simple indictment: start 
with practices! Ways of doing things form the key empirical unit of analy- 
sis, which researchers inductively study. The main goal is to identify the 
key practices that structure the political site at hand (i.e., a given IO) and 
understand their structuring effects. Identifying them requires an inductive 
approach that pays equal attention to established and deviating patterns. 
Practice analysis also necessitates going beyond formal procedures, even 
though most of what global actors do, in fine, does rely on forms of textual- 
ization and codification. 

In the interpretivist tradition, practices must be understood from within 
the community of practitioners so as to restore the intersubjective meanings 
that are bound up in them. This requires understanding the logic of practi- 
cality that organizes interaction. In order for practice X to do something in 
and on the world, what tacit know-how would practitioners need to have 
in order to grasp what is going on? In reconstructing practical knowledge, 
the objective is to understand the insider meanings that agents attribute to 


328 International Organizations and Research Methods 


their reality. Thanks to induction, the researcher refrains as much as possible 
from imposing scientific categories, to instead recover practical meanings 
and locally enacted commonsense. 

The key task amounts to the empirical redescription, as faithfully as pos- 
sible, of what is actually happening on the ground taking the practitioners’ 
point of view. This requires deep immersion and interpretation. After all, 
the generative power of practices stems from the meaningful context within 
which they are enacted, which instructs actors about what is going on. At 
the level of action, the meaning of the practice gets negotiated, in a more or 
less articulate fashion, between practitioners: what is happening here? Most 
of the time, rich interpretive clues are supplied by the existing intersubjec- 
tive context, which renders the negotiation process not all that elastic. For 
the social scientist, the basic objective is to understand what the practice 
under study counts as in the situation at hand. The methodological impli- 
cation should be clear: if the social efficacy of practices rests on the mean- 
ings that are bound up in them, then any social scientific account must go 
through the interpretation of social contexts and practical logics. 

As an inductive, interpretive, and historicist methodology, practice trac- 
ing admits a variety of methods, several of which are covered earlier in this 
handbook—Observe, Interview, Document, and Measure. For example, 
various forms of interviews (ethnographic, biographic, prosopographic, sem- 
istructured), discourse analysis (of documents, archives, speeches, images), 
and participant observation (direct or indirect, shadowing) are all possible 
options, depending on access and research objectives. What matters is to get 
closer to practitioners’ experiences so as to understand everyday life inside 
IOs. For that reason, the method of choice often is ethnographic participant 
observation, which involves the researcher’s direct and sustained participa- 
tion inside of a social setting and its everyday dynamics. The unique value 
added of this method, on top of allowing direct observation of practices, is 
that it usually takes place within the natural habitat of practitioners, with 
limited disturbance from the outside. 

In the actual practice of research, though, participant observation is 
often not feasible, whether for financial, organizational, legal, geographi- 
cal, or historical reasons. The good news is that, even when practices can- 
not be seen, they may be talked about through interviews or read thanks to 
textual analysis. Tracing practices can be done in a variety of indirect ways. 
For instance, where practitioners are alive and willing to talk, qualitative 
interviews are particularly apt at reconstructing the practitioners’ point of 
view. As conversations generative of situated, insider knowledge, interviews 
provide researchers with an efficient means to penetrate more or less alien 
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lifeworlds. The main challenge, however, is that contrary to representational 
knowledge, which is verbalized and can be brandished, practical knowledge 
is generally unsaid and mostly tacit. The solution is to focus less on what 
interviewees talk about than what they talk from—the stock of unspoken 
assumptions and tacit know-how that ought to be presumed in order to say 
what is being said. 

In order to turn interviewees into participant observers of sort, one may 
ask interviewees to recount in detail how they and their counterparts go 
about their business—what their daily schedule looks like, with whom they 
meet regularly, the kinds of negotiations they conduct, and so on. In dis- 
cussing everyday whereabouts with interviewees, the objective of is to get 
indirect access to the action floor, by recounting lambda workdays, standard 
procedures of operation, routine practices, or conversely, unusual stories 
and extraordinary anecdotes. Interviewees generally proved quite willing to 
describe their everyday life with a significant amount of detail. 

When practitioners are not available to talk, textual analysis can be put to 
work in order to trace practices and interpret the context in which they are 
performed. It is critical to select particular textual genres—those that record 
enacted practices, especially—in order to open up a window onto practical- 
ity, from memoirs to court cases through handbooks. Other useful genres 
include annual reports, diplomatic cables, meeting minutes, personal dia- 
ries, recordings and transcripts, written correspondence, and so on. Need- 
less to say, studying practices through written documents is easier said than 
done, because the more informal ones are generally left unsaid. Ultimately, 
a combination of sources—ethnographic, conversational, and textual—is 
probably the best way to draw a full portrait of the practice landscape under 
study. 


What Challenges? 


Practice tracing is full of challenges. First, in order to trace practices, one 
must find them. There are different ways to go about this and whichever 
strategy works best depends on the nature of the social site under study. 
Happily, IOs form relatively closed and circumscribed sites, to the extent 
that headquarters are geographically located at and around a few multilateral 
hubs distributed around the world. That said, IOs’ activities often extend 
across a variety of transnational networks whose ramifications often span the 
entire globe. Also problematically, several IOs are rather hard to penetrate as 
a researcher. This obviously has to do with confidentiality and the sensitiv- 


330 International Organizations and Research Methods 


ity of diplomatic matters, especially with real and consequential negotia- 
tions unfolding in the background. The world of multilateral diplomacy and 
global policymaking tends to be highly secretive, not least because informa- 
tion is power in the everyday life of a permanent representative. The prob- 
lem is not just that access to the action floor is severely restricted. In fact, 
the difficulty is compounded by the fact that, even where access is granted, 
stringent confidentiality agreements often significantly reduce the value of 
the data that may be published. 

Second, the identification of practices may run into difficulties. On 
the one hand, the issue of scale is always lurking: at what level of aggre- 
gation should the analysis operate? Should the researcher describe micro 
activities—e.g., how particular committee meetings function—or is it pref- 
erable to zoom out on broader patterns of action, such as coalition-making 
for instance? Finding the proper scale depends on the research question 
being asked. Some problems are best addressed by reconstructing minute 
interactions, whereas others require a more macro perspective. On the other 
hand, scaling is connected to another challenge, that of naming the practices 
under study. In general, it is preferable to hinge on practical categories, that 
is, replicate the language that practitioners use in describing their activities. 
‘This is easier to do with granular activities, several of which already have tan- 
gible existence for actors (e.g., “I was holding a bracket in the proposed text 
when this and that occurred”). As soon as the researcher scales up practices, 
however, more work is needed in labeling broader patterns, which do not 
necessarily exist as such for practitioners. Superimposing alien categories on 
practices is acceptable so long as it serves analytical objectives. 

Third, getting at practical logics presents daunting challenges. In order to 
decipher the meaning of a practice, its practicality must be both alien and 
native to the interpreter’s own system of meanings. If, on the one hand, the 
meaning of a practice is too deeply embodied by the interpreter, chances are 
that it will remain invisible as a second nature. If, on the other hand, the 
logic of practicality is completely alien to the interpreter, then it may not be 
properly understood within its context. Epistemically speaking, the researcher 
is sitting on the fence between the community of practitioners and that of 
researchers, a position that generates a form of knowledge that is at once native 
and foreign. As ethnographic or inductive as one may go, studying practices 
implies ordering, dissecting, and organizing them in a way that ultimately 
constructs them as units of analysis within an analytical narrative. Instead of 
perfect correspondence, the objective should be to capture practical logics so 
as to explain their social effects, bearing in mind the reconstructive process 
that the interpretation of meanings necessarily entails. 
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Fourth, practices are by nature polysemous, contested, and multiple. 
Because the everyday is an inexhaustible stream of data, studying practices 
means inventorying an infinite pool of ways of doing things. It is never pos- 
sible to brush an exhaustive portrait, from a practice perspective, or to cap- 
ture everything that goes on that matters. This kind of maximalist notion of 
data gathering is problematic and potentially paralyzing. Many diplomatic 
practices are so casual—who plays tennis with whom, who goes out to smoke 
during the break, and so forth—that it is plainly impossible to achieve any 
kind of exhaustiveness. Sometimes, focusing on the lambda habitus—the 
common stock of tacit know-how from which variations spring—is the only 
practicable road. Carefully drawing a sample of practitioners, sites, and areas 
of action helps cover a variety of national viewpoints, agencies, career ech- 
elons, and professional backgrounds, in order to identify convergences but 
without disregarding discrepancies. Ultimately, practices and practical logics 
are always plural, especially in IOs where there is a ton of circulation across 
networks and organizations. 
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CHAPTER 28 


Feminist Approaches 


Georgina Holmes 


International organizations (IOs) are not gender-neutral but complex, fluid, 
and changeable gendered systems shaped by the politics of exclusion and 
inclusion. Disrupting mainstream analyses in IO studies, scholars apply 
diverse and creative research methods and methodological approaches to 
examine how institutional change processes take effect in and through 
gendered power relations. The aim is to expose and explain the political 
consequences gendered processes have within IOs and within international 
society, while being mindful that formal, institutionalized equality and 
diversity initiatives can potentially strengthen existing institutional power 
settlements. 


What? 


The creation of feminist (and multiple) knowledges about IOs requires criti- 
cal and creative approaches to access, gather, and analyze data. Research 
methods used by feminists, gender scholars, and queer scholars are not in 
and of themselves “feminist” and/or “queer” and any research method may 
be used as a tool to collect data about gendered IOs. Rather feminist and/or 
queer methodologies determine research strategies and the ethical decisions 
made by researchers. Connell conceded early on that “the complexity of 
organizations, and the complex character of the gender system [has] meant a 
single research strategy [is] inadequate.” Connell also expressed concern that 
quantitative research alone—for example, statistical analysis of the gendered 
division of labor—cannot explain gendered power dynamics within com- 
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plex, open, and dynamic IOs (Connell 2005: 4). This is particularly the case 
when trying to map an IO’s gender regime. Feminists have therefore tended 
to adopt either qualitative or mixed methods approaches to study IOs from 
the bottom up. 

In the early 1990s, Acker challenged the normative view that “gender” was 
a singular, binary category (male/female) and argued that gender constituted 
an ordering principle within institutions (Acker 1992). International orga- 
nizations were dominated by men, but their organizational structures were 
not gender-neutral, as many feminists at the time perceived. For Acker, IOs 
were sites marked by women and other minority groups’ absence and mar- 
ginalization, but also sites of multiple and intersectional forms of structural 
oppression and violence, including racial, sexual, and class-based. Defined 
by the politics of inclusion and exclusion, gendered processes produced and 
sustained institutional power hierarchies, institutional cultures and sub- 
cultures, and governed the social actors operating within them. Raewyn 
Connell coined the term “gender regime” to describe the “patterning” and 
“continued patterning of gender relations” within an organization. An entire 
IO has a gender regime, but gender regimes may also be present within “a 
particular site within an institution” and “can be applied in a parallel way to 
an organizational process such as policy formation.” For Connell, the task 
of researchers investigating how IOs are gendered and their gendered effects 
is to “understand and illuminate distinctive patterns in organizational life” 
(2005: 7). This has required creating new feminist knowledges about IOs by 
deconstructing what appears to be normal, naturalized, and often invisible 
within them (Gherardi 2009). 

Feminist analyses of IOs can be found in several fields including devel- 
opment studies, organizational studies and management studies, political 
science, international law, and more recently feminist security studies, but 
gendered IOs are still an understudied area of research. In the early 1990s, 
feminists were influenced by the liberal feminist focus on women’s equal 
opportunities, integrating women into IO workforces and incorporating 
women’s issues into IO external programming. For these feminists, IOs were 
not international and failed to promote universal human rights because they 
did not represent at least 50 percent of the world’s population (Charles- 
worth 1995: 79). Developing case studies about the World Bank, United 
Nations Development Programme, International Labour Organization, and 
United Nations Security Council, feminists began to investigate how gender 
mainstreaming—a policy tool designed by gender practitioners in the devel- 
opment and aid sectors—was being institutionalized within IOs to readdress 
gender imbalances in organizational cultures, structures, policies, processes, 
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and programming (see Priigl and Lustgarten 2006; Hafner-Burton and Pol- 
lack 2002; Charlesworth 2005). 

In parallel, feminist security studies scholars investigated the gender pol- 
itics of global governance, largely influenced by agency-structure debates 
within the field of international relations; actor-network theory; con- 
structivist debates on the emergence, diffusion, and institutionalization of 
international human rights norms; and Foucauldian-inspired literature on 
governmentality. Drawing on conceptual and analytical tools developed by 
feminist institutionalists within political science, feminist security studies 
scholars began to examine how the international women, peace, and security 
agenda was implemented and how security policies were gendered and have 
gendered effects, prompting analyses of international and regional security 
institutions including the UN Security Council, NATO, and the European 
Union (Guerrina and Wright 2016; Basu 2016; Chappell et al. 2020). What 
motivates IOs to engage with gender is a central question, although femi- 
nists continue to seek out the ambivalent, the contradictory, and the para- 
doxical within IOs. 


Why? 


Soumita Basu suggests that gendered analyses are important because IOs are 
“arenas of emancipatory politics” (2019). Contemporary IOs were estab- 
lished in the post-World War II era to ensure global populations could exer- 
cise their human rights and carry out their everyday lives free from all forms 
of discrimination and violence. However, they also protected the status quo 
among major powers, and protected sovereignty more broadly. Decoloniza- 
tion in the 1950s and 1960s, which saw the inclusion of new state actors 
within IOs, was to be one outcome of this aim. Feminist scholars are there- 
fore driven by a political project to redefine the power settlement within IOs 
and facilitate the design and maintenance of “gender-just” institutions, ide- 
ally by adopting an intersectional approach. Increasing the proportionality 
of all women and minority groups within IOs—for example, lesbian, gay, 
bisexual, transgender, and questioning (LGBTQ+) persons—redistributing 
institutional power and exposing and eradicating all forms of discrimination 
and violence support these aims. This requires investigating the paradoxes 
and ambivalences of gender regimes as both violent and emancipatory; 
examining how feminists, gender experts, and their critical friends operate 
within IOs; monitoring institutional change and assessing whether insti- 
tutionalized gender reform agendas are transformative and actually redress 
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power hierarchies and institutional rules, norms, practices, and cultures 
(Prügl and Lustgarten 2011). 

Mainstreaming gender equality and diversity is still a relatively new arena 
of politics within IOs. The United Nations first began to take what was 
then called “the Woman Question” seriously in 1975 (Bessis 2003: 636). 
The Convention on the Elimination of all forms of Discrimination against 
Women was adopted in 1979. Women’s rights were further institutionalized 
after the Beijing Platform for Action in 1995, the Millennium Development 
Goals, and the adoption of United Nations Security Council Resolution 
1325 on Women, Peace and Security in 2000. Although at the time these 
policies were progressive and a major success, gender scholars and queer 
theorists have levied criticism against the Western, liberal feminist approach 
to mainstreaming gender equality, with its emphasis on a singular and sta- 
ble category “woman,” and subsequent neglect of how a diversity of people 
(assigned categories within the institution and broader society according to 
sex, gender identity, sexuality, age, race, ethnicity, class, ableism, and so on) 
experience intersectional forms of institutionalized discrimination and vio- 
lence (Ahmed 2012; Hagen 2016). 

However, there are still important areas that require investigation includ- 
ing the transnational and transregional dimensions of IOs; exposing how 
competitions and partnerships between different patriarchal regimes within 
the international system manifest in and through IOs and the effects these 
relationships and tensions have both on creating gender-just IOs and inte- 
grating gendered and inclusive approaches into IO external programming; 
and mapping more systematically structural changes that are taking place in 
IOs over time and temporality (Holmes et al. 2019; Holmes 2020). 


How? 


Scholars who are not feminists often take inspiration from the creative 
approaches feminists develop to gather gendered knowledge about IOs. 
However, one of the first steps any scholar can take is to pay attention to 
how their gender or perceived gender identity might affect their access to 
IOs, data collection processes, and data analysis. Feminist ethnographic 
methods, including participant observations, in-depth interviews, focus 
groups, life history interviewing, and surveys are among the most com- 
mon research methods used to examine formal and informal IOs. Feminist 
ethnography—the study of the “dynamics of power in social interaction 
that starts from a gender analysis’—emphasizes the experiential and draws 
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strongly on the feminist tradition of centring the bodies and experiences 
of institutional actors in analyses (Davis and Craven 2016: 9). Emphasiz- 
ing the performativity of gender and connections between the body and 
embodied knowledge and observing the partial, situated, and intersubjec- 
tive tensions at play in data collection, feminist ethnography aligns with 
Wacquant’s carnal sociology (see box d—Carnal Sociology) but focuses on 
how carnal sociology is itself gendered and produces gendered embodied 
knowledge, as well as gendered representations and affects. More attention 
is also paid to centering research participants and achieving equity between 
researchers and research participants than during mainstream positivist IO 
research. Techniques that feminists might use here include engaging research 
participants in the research design, feeding back research findings to research 
participants, and being attentive to the power dynamics at play during field 
research—for example, by reflecting on how the researcher’s own embodied 
presence and experience within an IO might influence the kind of research 
data obtained (Davis and Craven 2016: 84). 

Another approach is to gather institutional artifacts, documents, and 
discourses. Institutional artifacts may include formal, publicly available 
policies, speeches, press releases, promotional materials, and other grey lit- 
erature, as well as restricted and confidential institutional documents such 
as internal policies, procedures, and training materials (see box o—Artifact 
Analysis). Publicly available institutional artifacts are obtained by conduct- 
ing archival and desk-based research. 

A third approach is to collect data in order to record and construct prac- 
tices within institutions, or to identify formal and informal rules and norms 
that govern actors within IOs (see chapter 27—Practice Analysis). For exam- 
ple, feminists examine how gendered, embodied practices are performed by 
people who engage with IOs—either as employees and employers or as out- 
sider actors who interact with them; how practices become institutionalized 
and embodied within institutions; and how actors working within institu- 
tions seek to disrupt or change institutional practices (see Bode 2019). 

Each strand of feminism—whether it be radical feminism, Marxist 
feminism, psychoanalytical feminism, poststructural feminism, postcolonial 
feminism, and queer studies—adds new perspectives to the study of gen- 
dered IOs and determines research design (Gheradi 2005). The positionality 
of women, in particular women of color, women from the Global South, 
LGBTQ persons, and other minority people within the international system 
has meant they have been required to design and conduct research projects 
at the margins of IOs, global politics, and their academic disciplines (See 
Holmes et al. 2019). One approach, suggested by Basu, is to have multisited 
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analysis of IOs. Here researchers do not just undertake research within the 
headquarters of IOs but explore what the IO looks like from different loca- 
tions around the world. Applying postcolonial and critical race theories (see 
box x—Postcolonial Insights) to examine global power relations within IOs 
helps to complicate what is meant by the “marginalized” within IOs, and it 
is crucial to observe the contradictions and tensions within and between the 


groups of institutionalized actors which the researcher identifies as “subal- 
tern” (Holmes et al. 2019). 


What Challenges? 


Accessing IOs is a major challenge for feminist researchers, in part because 
gender politics are often regarded as very sensitive and data is often restricted. 
Often the process of networking, gaining the trust of institutional gatekeep- 
ers, and obtaining permissions to conduct field research can take months or 
years. Institutional gatekeepers may attempt to influence research design, 
co-opt research agendas, or prevent access to institutional actors who they 
perceive may threaten to undermine official institutional gendered dis- 
courses about IOs. Working in the capacity of a critical friend is one strategy 
frequently adopted, although the ethical issues and effectiveness of this strat- 
egy are widely debated (see Chappell and Mackay 2021). Collaborations 
with institutions are another approach, though as Connell observes they can 
be costly, time consuming, and require the researchers to make compromises 
(2005). Institutional gatekeepers may have particular expectations about 
how the outsider-researcher should perform within an IO and researchers 
are mindful of how their own bodies may or may not “fit” an institutional 
culture and may prevent them from building relationships with institutional 
actors or from accessing alternative stories and narratives. Feminist, gender, 
and queer scholars are also acutely aware that their positionality may prevent 
them from observing certain practices, whether violent (such as casual rac- 
ism) or emancipatory. 

A second related challenge concerns how to investigate the informal, the 
invisible, the silenced, and the unspoken about within IOs, which in turn 
raise ethical concerns around ensuring that research participants who con- 
fide with the researcher are not exposed. Formal rules, norms, and practices, 
or indeed gatekeepers, may prohibit researchers from using some of the 
more creative feminist ethnographic research methods, such as producing 
artwork and making photographic or video diaries that are used in collabor- 
ative and community-based research projects. Institutional sensitivity con- 
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cerning gender issues within IOs and logistical challenges therefore compel 
researchers to think creatively about how to access and analyze research data. 


To Go Further 
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BOX X 


Postcolonial Insights 


Soumita Basu 


The passage of UN Security Council Resolution 1325 on Women and Peace 
and Security (WPS) in 2000 is widely considered to be historical, as this 
was the first substantive engagement of the Council with issues relating to 
women and armed conflicts. The nature of its implementation, selective, 
underfunded, and directed primarily toward conflict-affected countries in 
the Global South but driven largely by actors from the Global North, lend 
credence to concerns that the resolution—and later its sister resolutions— 
were “reinscribing racial-sexual boundaries in international security” (Pratt 
2013; see also Parashar 2018). These concerns draw on and resonate with 
postcolonial critique in international relations. Such critique “has served as 
a deconstructive tool challenging universalizing discourses of colonialism, 
modernity and nationalism (even in its anti-colonial form)” (Duvall and 
Varadarajan 2003: 84). In the study of IOs, extrapolating from scholarship 
on global governance (see Muppidi 2005), this can draw attention to “the 
negotiation and regulation of difference” (274), as well as the ways in which 
the structure, policies, and practices and indeed our imaginations of IOs 
constitute some as “the governors,” with agency, and others as “the gov- 
erned” (280). Discourse analysis that focuses on power-knowledge nexus, 
often conducted using textual analysis, is well-suited for postcolonial inves- 
tigations that spotlight such hierarchies. In ethnographic research, it can be 
valuable to decenter fieldwork, studying not just headquarters and confer- 
ence sites but also “subaltern” spaces within institutions as well as periph- 
eries where IO policies manifest (Holmes et al. 2019: 220). More gener- 
ally, giving due consideration to the postcolonial question about “whose 
knowledge” matters, as reflected in the study and practice of IOs, can enrich 
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the field by engaging with diversity. This can be factored into selection of 
interview respondents and primary documents, as well as citation of second- 
ary literature. For instance, advocacy papers and reports of local women’s 
peacebuilding organizations could provide context-specific critique of inter- 
national organizations and their policies alongside proposals for alternative 
formulations. Research choices of this kind would recognize and potentially 
reinforce the agency of the “governed,” challenging prevalent hierarchies of 
knowledge production. 
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BOX Y 


Reflexivity in Practice 


Audrey Alejandro 


Researchers are commonly encouraged to be reflexive about the knowledge 
they produce and their positionality. However, they often find themselves at 
a loss when it comes to implementing reflexivity in practice due to the lack 
of adapted methodological material. 

To address this challenge, I developed Reflexive Discourse Analysis 
(RDA) as a methodology for reflexivity. RDA “aims to reflexively assess and 
transform our socio-discursive engagement with the world, so as to render it 
consistent with our intentional socio-political objectives” (Alejandro 2021: 
3). I define discourse as language in context and reflexivity as the practice of 
making conscious and explicit our practices, beliefs, and assumptions (see 
also chapter 11—Discourse Analysis). 

RDA comprises three components: 

First, you create a “compass-discourse,” by which I mean a sentence that 
summarizes an alternative to the biases you aim to avoid in your discourse. 
For example, researchers may inadvertently produce unethical or invalid dis- 
courses regarding the agency of so-called local actors (e.g., the beneficiaries 
of IO policies) by either denying their agency or romanticizing their role. 
To address this issue, I suggest using the following compass-discourse: “all 
agents connected through IO interventions potentially play a role in the pro- 
duction, modalities and outcomes of these interventions, which cannot be 
assumed without being investigated.” 

Second, you empirically assess your writing—e.g., drafts—via discourse 
analysis to see whether the implicit dimensions of the discourses you pro- 
duce match your compass-discourse. To investigate whether you inadver- 
tently deny the agency of so-called local actors, you may, for example, look 
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for subject positionings (e.g., who are the subjects in my text? What are the 
adjectives and verbs associated with these subjects?), and processes of pas- 
sivization (i.e., the transformation of potentially active sentences into passive 
ones leading to the discursive disempowering of the actors being passivized).! 

Third, you engage in self-resocialization: you identify and transform the 
social dispositions that led to this problematic discourse. The objective of 
RDA is not about switching a few words in an effort to be more politically 
correct; rather it is about using textual material to empirically assess your 
discourse as an entry point for long-term social transformation. Denying the 
agency of those we write about can enable us to position ourselves as their 
savior, potentially satisfying our own interest to their detriment. It may also 
reveal that we project our own emancipatory quest onto their journey. 

RDA is an iterative process: go back and forth between DA and the 
transformation of your dispositions and repeat the steps until the implicit 
dimensions of your text match the intentional sociopolitical objectives you 
have set up for it via your compass discourse. Reflexivity is not easy, but it is 
far from impossible! 
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CHAPTER 29 


Composing Collages 
Working at the Edge of Disciplinary Boundaries 


Anna Leander 


Is it possible to think of methods in a way that does not turn them into a 
Procrustean bed for knowledge, cutting off any body part extending beyond 
the frame while stretching those that don’t reach it? This chapter suggests 
that the answer to this question is yes. It is possible to work with methods 
not as instructions but as “heuristic devices” (Abbott 2004). The fundamen- 
tal question is not “does the method allow this?” but rather “what/how does 
it help me discover?” Methods do not have to work like recipes in a cook- 
book. They can be “thinking tools” (Leander in Klotz and Prakash 2008). 
This chapter introduces one such thinking tool associated with an affirmative 
understanding of methods: the Composing of Collages (Leander 2020). 


What? 


In the humanities and critical social sciences (including decolonial, gender, and 
racial studies, see chapter 28—Feminist Approaches, and box x—Postcolonial 
Insights) methods have a poor reputation. They are charged with stifling 
observation, thought, and creativity by regimenting observations, disciplin- 
ing knowledge, and providing a false sense of scientific security where none 
is warranted. They reinforce the regressive politics of science Rosana Paulino 
criticizes in her collage Amor pela Ciencia (fig. 12). Refusing methods—often 
by opposing them to theorizing—has therefore been the default position for 
the longest time and is still the norm in some areas including, for example, 
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Figure 12. Rosana Paulino: Amor Pela Ciencia. Print and embroidery on textile 
Source: Courtesy of Rosana Paulino 


political theory, the humanities, or visual studies. However, in most quarters, 
including the study of international organizations (IOs), this refusal tends to 
become a form of self-exclusion as the ability to articulate a methodology has 
become the sine qua non condition for doing research. Often therefore there 
is little choice but to own “methods” and develop a “pluralist” language for 
talking about them (Bleiker 2015). But how? Working with methods as Com- 
posing Collaborationist Collages is one avenue. 

The term composing emphasizes that methodological choices constitute, 
define, and frame the observed (Austin 2019). We see what we look at as 
we decide which “paths” to follow in our research (Ranciére 2009; Mitchell 
2015). When we make methodological choices regarding which sources to 
focus on and how to deal with them, we are also “composing” our object of 
observation. When deciding what to include in our research on IOs we are 
composing a specific image of them. 

The term collage signals an openness to variety and disjuncture. Collages 
are by definition made of heterogeneous materials and consequently draw 
on an open-ended variety of techniques to connecting these. When we work 
with IOs, there is a wealth of possible materials at our disposal including, 
for example, academic publications (secondary sources), documents, obser- 
vations, interviews, visuals, smells, tastes, sounds, atmospheres, affects, con- 
cepts, and so on. The techniques and tools helpful for exploring them vary 
correspondingly as do the images generated by joining them. 

Composing collages to explore IOs cultivates knowledge directing careful 
attention to (dis-)connections in view of making theoretical and empirical 
discoveries and contributions. It defies the disciplinary policing that ham- 
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pers cross-disciplinarity and novel agendas and research geared at discovery 
tout court by imposing questions, criteria, sources, and tools. The Compos- 
ing of Collages actively invites and encourages research that draws inspira- 
tion beyond their own disciplinary boundaries and questions the boundar- 
ies of knowledge. Composing Collages is a method for doing disciplinary 
“edgework,” to speak with Wendy Brown (2009). 


Why? 


Contributing knowledge by working at the edge of disciplinary boundaries 
sounds like something most methods would claim to facilitate. Four caveats 
connected to the radical openness of the Composing of Collages, and the 
relevance of this for the study of IOs, clarify the difference Composing Col- 
lages makes. 


(i) Composing Collages is neither regimented by a specific “analytical 
framework” that posits what questions to ask nor is it confined by 
the regime of knowing imposed by it. This is particularly important 
for raising questions not already raised. For example, work on gen- 
der, race, or material agency that is now mainstream in IO research 
has only been possible because scholars did not “apply” meth- 
ods associated with prevailing analytical frameworks, but instead 
mobilized methods to answer their own questions and often also to 
transform these questions. They simply refused to answer the “what 
is this a case of?” question. They were interested in problems that 
were not “cases” of prevailing theories and methods but obscured 
and deepened by them. This was/is no abstract consideration as the 
black woman in Paulino’s collage “love of science” recalls (figure 
12). Strict “academic rigour” would have meant “rigor mortis” for 
the research agendas around class, gender, race as it tends to mean 
for innovative research agendas generally (Czarniawska 2016). 
Composing collages—mobilizing methods as heuristic devices—is 
a way of eluding this fate as it is associated with a decidedly “coun- 
terdisciplinary” (Koskenniemi 2012) view on research. 


(ii) Composing Collages allows multiple, messy ontologies (Law 
2004). Contrary to many methods, it therefore does not assume 
that a singular, universal epistemic logic or world is pertinent for 
research (de la Cadena 2015). Rather, as “quantum physics” or 
“new materialist” anthropology, it admits the existence of mul- 
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tiple, contradictory, and perhaps incommensurable logics to be 
at work at the same time and in the same space. In IOs, multiple 
cultures and beliefs coexist. More strongly, so do different onto- 
logical “‘realities.” In IOs, different professional practices inhabit 
partially connected worlds irreducible to each other that produce 
different policy initiatives. Composing Collages is particularly 
useful for shedding light on the relations and frictions between 
these multiple worlds and the implications of this for IOs. 


(iii) Composing Collages is also open to a wide range of sources and 
data. Many methodologies provide strict instructions for how to 
regiment what counts as data. Working with Composing Col- 
lages is different in this respect in that, in line with anthropology, 
it invites combinations of sources (Latour 2005; Czarniawska 
2007; Ingold 2008). For IO research, this openness is interest- 
ing for heuristic reasons. It encourages us to explore a variety of 
sources—images and/or sense-making beyond language (Mitch- 
ell 2015; Howes and Classens 2014; Introna 2018; see also box 
d—Carnal Sociology)—either in relation to existing lines of 
research or to generate novel ones. More than this, the openness 
to heterogeneous sources, and the pragmatism this allows for in 
terms of what is required for any specific study, is appealing to IO 
researchers constantly negotiating restrictions stemming from, 
for example, political sensibilities, confidentiality, or limits to 
access/ participation. 


(iv) Composing Collages is not limited to a specific form of research 
communication. On the contrary, it connects the heterogeneous 
in shifting ways. It can move beyond the narrative not only to 
the lyrical (Abbot 2007) but also to images, videos, objects, and 
beyond. One might imagine also exhibitions or performances. 
For IO researchers who often have the ambition to convey their 
research results to an audience beyond the scholarly, this open- 
ness has considerable advantages and remains a reason for work- 
ing with the approach. 


How? 


Is it possible to provide “instructions” for how to do counterdisciplinary 
research by Composing Collages? The answer is (paradoxically?) yes, pro- 
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vided the instructions are commensurate to the ambitions. Four ows appear 
particularly central to the Composing of Collages: 


(i) Establishing a fit between research focus and method: For the Com- 
posing of Collages, this is particularly challenging. The method 
does not come with predefined criteria for how to do this. Rather 
researchers are asked to establish the fit thinking it through, possi- 
bly adjusting it in the process of doing the research. This is unset- 
tling for those who think that methods are there precisely to fix 
this fit. However, it is also an advantage. Research is done to dis- 
cover things not already known. To revisit assumptions about the 
fit between focus and method once basic “cultural competence” 
is acquired is therefore a considerable advantage for any research 
that explores something not already known (Neumann in Klotz 
and Prakash 2008). More affirmatively, mobilizing methodologi- 
cal choices, combining and crossing them, can serve as a “heuris- 
tic device” generating research foci (Abbott 2004). 


(ii) Working imaginatively: The radical openness of Composing Col- 
lages leaves little choice but to exercise methodological imagina- 
tion, pragmatically adjusting to and working with the ready-made 
and available in each given context. Imagination is called for in 
the engagement with sources, analytical tools, concepts, theo- 
ries, and writing/communicating scientific findings. This makes 
research creative and fun (Haraway 1997). The risk that it deto- 
nates the disciplinary canons is steadily diminishing. The expand- 
ing space made for imaginative forms of work and the pressure 
to prove “impact” and “relevance” conspire to locate imaginative 
forms of research and research communication more acceptable. 
For instance, exhibitions are increasingly part of scholarly com- 
munication, for example, about the governance of human rights 
and torture (Jasanoff 2017; Austin 2019). 


(iii) Flattening Epistemological and Ontological Hierarchies: To work with 
Composing Collages allows and requires a flattening of the hier- 
archies that permeate research. At the core of the Composing of 
Collages sits the idea that the best way of providing a grasp of the 
world is not to construct an empirical generalization or a general 
explanation even if delimited by “scope conditions” but to accept 
that also concepts and theories are situated and inexact. Collaging 
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therefore strives to give voice to and show the observed on its own 
terms. To do this demands making space for the nontheoretical 
and nonacademic, that is for nonacademic practices, images, sense- 
making, resonances, atmospheres, artifacts, documents, filing prac- 
tices, track-change software, big data, tracking technologies, and 
beyond. To Compose Collages is to collaborate or colabor with the 
observed to gain knowledge rather than simply “‘objectifying” it 
(Bourdieu et al. 1991; Haraway 1997; de La Cadena 2015). 


(iv) Checking Positionality through Frictions: Logically following 
from the above, as anyone acknowledging the situated nature of 
knowledge, those Composing Collages must grapple with the 
“positionality” of the researcher and its consequences for what 
can and will be observed, understood, analyzed, and written 
(see box c—Multipositionality, and box x—Postcolonial Insights). 
“Reflexivity’—that is reflection on one’s own research practices— 
remains crucial in Composing Collages. However, in addition to 
reflexivity collaging relies on the frictions generated in collabo- 
ration as a check on positionality. The flattening of hierarchies 
and mobilization of a diversity of materials provide information. 
More significantly, they place the research in perspective. Instead 
of assuming that positionality is best dealt reflexively, by looking 
into a mirror, Composing Collages mobilizes frictions, as Pau- 
lino shows when juxtaposing scientific texts about race with the 
embodied black woman (fig. 12). The “disturbances” generated 
in the encounters with heterogeneous others marks the points 
where positionality matters for research (Czarniawska 2007; also 
Bourdieu et al. 1991; Law 2004). 


What Challenges? 


Composing Collages is a methodology geared to support discovery, imagina- 
tion, and creativity. These are values professed by most researchers. However, 
they are also values exceedingly difficult to practice in real academic life (and 
beyond). Academia is inherently conservative. It is the guardian of authori- 
tative knowledge. For good reasons, academic institutions see themselves as 
bulwarks against fads, fashions, fabrications, and fakes. This makes them 
suspicious of claims to novelty and calls for change. Academic institutions 
therefore often generate processes where research that disturbs established 
truths, orders, and hierarchies is unwelcome, actively discouraged, marginal- 
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ized, and if possible buried (Stengers 2000; Law 2004; Csarniawska 2016). 
Methods are core to academic disciplinary disciplining. Innovative method 
(by definition!) fails to faithfully “‘apply” analytical frameworks, “conform” 
to “the cannon,” and “live up to” established “standards,” “criteria,” and 
“expectations.” It breaks and transgresses them. As it departs from authorita- 
tive knowledge, innovative research also necessarily challenges the academic 
foundations and guardians of that authority. Researchers practicing such 
methods (and this includes composing collages) therefore balance on the 
narrow ridge separating disreputable impostors and respected researchers. 
Their work is on the edge and must remain there (Brown 2009). The alterna- 
tive is submitting to the conservativism of conventional academia and rein- 
forcing it on the one hand or driving the dissolution of academic authority 
exposing all research to manipulations, markets, and solipsistic manias and 
turf constructions on the other. The peculiar “edgework” required to remain 
in balance on the ridge poses three challenges to the Composing of Collages. 

‘The first is internal to the research: the challenge of justifying meth- 
odological choices without reverting to disciplining disciplinary criteria. 
This assumes a methodological literacy and fluency considerably higher 
than what is needed for those who resort to ready-made instructions. Yet 
in many contexts little is done to nurture such literacy. Methods training is 
mostly narrowly focused on the quantitative, statistical, and formal, leaving 
researchers ill prepared. To become autodidact is a time-consuming commit- 
ment made at the cost of research. Better, broader, and more varied method 
conversations are much needed. (This book is a step in that direction.) 

‘The second challenge is related to the outward communication of meth- 
ods choices to externals who have a say over, are involved in, or are con- 
cerned by the research. This includes among others peers, students, scholars 
from academic specialties beyond one’s own, but also obviously the profes- 
sional world, like IO practitioners, and wider society. All have more or less 
articulated assumptions and expectations about “methods” (Jasanoff 2017). 
Conveying methodological reasoning that breaks with these unarticulated 
assumptions requires researchers to master their own reasoning, as well as 
locating it in relation to the prevailing “common sense” of others. 

The third, most significant, challenge is therefore posed by the tempta- 
tion to “pragmatically” take ready-made shortcuts and/or surrender to the 
nostalgia for firm foundations. Giving way to this temptation throws the 
researcher off the edge, back into bolstering the conservativism of academic 
conventions, and back into “the science” Paulino’s collage criticizes (figure 
12). To avoid this, there is little choice but that of tackling these challenges. 
This chapter is an invitation to do precisely that. 
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BOX Z 


Expeditions as a Research Method 


Doaa Abdel-Motaal 


Expeditions as a research method are considered highly problematic in the 
contemporary social sciences because of their deep roots in imperial con- 
quest, extraction, and settler colonialism. And yet how can we know the 
world without exploring it? How can we study structures governing highly 
remote places without traveling there? Expedition and exploration are two 
sides of the same coin, and surely one must distinguish between scientific 
research expeditions and expeditions that were designed to conquer new 
lands, appropriate their resources, and subjugate their original inhabitants. 

Studying the earth’s extreme environments, without scientific expeditions 
would be severely limiting. There is no doubt that traveling to the Arctic 
and the Antarctic opens up a universe of new research questions. Research- 
ers can see their initial research questions completely upended when they 
experience firsthand the remoteness and isolation of the polar regions, the 
fragility of their flora and fauna, and interact with their inhabitants such as 
the indigenous communities of the Arctic, and the near permanent research 
communities and relatively few human settlers of the Antarctic. 

In Antarctica, the Battle for the Seventh Continent, a book published in 
2016 based on two scientific research expeditions through the Antarctic and 
the Arctic, this is precisely what happened. Despite years of research on the 
Antarctic, its changing climate, and the Antarctic Treaty System that governs 
the continent, the first landing on Antarctica’s shores turned my research 
questions upside down. For, no amount of readings can prepare travelers for 
what it means to arrive at a continent with no political borders, no immi- 
gration officers and therefore no passport controls. The simplest of research 
questions became the foundation for the book: what happens when a person 
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decides to live, die, marry, or give birth in Antarctica? In short, who owns 
the earth’s last remaining continent? 

Born in 1959, the Antarctic Treaty demilitarized the continent and dedi- 
cated it to peace and scientific research. However, it took decades (2004) 
for parties to create the Secretariat of the Antarctic Treaty, an international 
organization to administer the treaty, placing it in Buenos Aires (Argentina). 
Hailed as one of the greatest successes of the international system, it took 
a research expedition through Antarctica to understand that the Antarctic 
Treaty and its secretariat were just as fragile as the continent. In fact, the 
melting of Antarctica’s ice and the rising competition for its living space 
and resources, is a threat not just to the continent but also to its governance 
structure. 

Expeditions are an essential tool for the study of international regimes, 
and of the international organizations behind them, when they are intrinsi- 
cally linked to specific geographic locations. They become vital to under- 
standing whether governance structures truly respond to the needs of those 
regions, to their pace of change, and to the hopes and aspirations of their 
inhabitants. They can provide vital insight into the real questions that need 
to be addressed. In fact, the best expeditions are the ones that allow research 
questions to evolve, with a framing and multiple reframings of the problem, 
as the discovery and exploration proceed. 
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INTERLUDE V 


Controversies on Methodological Pluralism 


J. Samuel Barkin 


All research into IOs relies on a mix of methods. The chapters through- 
out this book show how to use a variety of methods to their full potential 
and in this section specifically how some methods can be combined with 
others to good effect. Yet no individual method suffices to answer research 
questions about international organization or international organizations. 
There are methods for getting raw information in the first place. One can 
do interviews, observe practices, read primary documents, study histories, 
or find handy data sets that someone else has done the work to compile. 
Entirely different sets of methods can extract data from that raw informa- 
tion. A review of the same set of primary documents, for example, can yield 
a historical narrative, a discourse analysis, or a set of quantified data points. 
‘These in turn can be understood through a variety of techniques. The his- 
torical narrative, for example, can be deployed as comparative case study or 
as genealogy; the quantified data set can be deployed as description or in 
various forms of models for statistical hypothesis testing. 

In other words, a complete methodology requires at minimum distinct 
methods for the acquisition, modeling, and analysis of information. What 
is often referred to as mixed methods in social science research uses different 
methods within one or more of these stages of research in specific method- 
ological contexts. But published IO research always mixes methods, whether 
inside or outside these contexts. Even the publication of a data set without 
accompanying analysis of the data, for example, involves mixed methods, 
insofar as the methods for acquiring the raw information and for quantify- 
ing that information are distinct. Saying that our methods are mixed or 
combined, then, tells us nothing new. The more interesting question is how 
we are mixing our methods. 
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Because the combining of methods is necessary to do social science, and 
therefore IO, research, the mere fact of doing so cannot be controversial. 
‘Therefore this essay, despite being in the part of this volume on combin- 
ing methods, is not about controversies in method combination but rather 
about controversies in methodological pluralism. The key shift here is not 
from questions of “combining” to questions of “pluralism.” It is more subtle; 
it is the shift from methods to methodologies. The bulk of this essay focuses 
on the relationship between these two concepts and will return to the issue 
of methodological pluralism at the end. 


What Exactly Is a Qualitative or Quantitative Methodology? 


I understand a method to be a specific tool—in this context a specific tool 
for the acquisition, manipulation, analysis, or contextualization of informa- 
tion relevant to the study of international organizations. These tools can be 
employed more or less effectively, and the various chapters in this volume give 
invaluable advice on how to apply many key social science research methods 
well. The most effective use of a tool is also context-specific, whether the tool 
is statistical modeling or interviewing or observing. Methodology can then 
be understood as “the strategy by which methods are selected, given research 
questions,” and, more broadly, “the way in which scholars think about the 
relationship between methods and research questions” (Barkin and Sjoberg 
2017: 4). Methods are the tools, methodologies are the ways in which the 
tools are utilized in combination to address specific questions about inter- 
national organizations. 

Methods are then quite distinct from methodologies (some authors in 
this volume refer to research designs; I take the term research design to be 
more or less equivalent to methodology). Furthermore, methods do not nec- 
essarily map onto methodologies. A good illustration of this claim can be 
found in the distinction often drawn, both within IO studies and across the 
social sciences, between quantitative and qualitative. When used to describe 
methods, this distinction might make some but limited sense. Statistical 
analysis, for example, seems pretty clearly quantitative (although it is less 
how one might characterize something like fuzzy set theory) (see, e.g., Ben- 
nett and Elman 2007). Formal rational choice models are less clear. They are 
mathematical, but they are not actually quantitative in the sense of using 
quantified data. Surveys are often used for statistical analysis but need not 
be. Historical research can be quantified or not; historical narratives can 
reference descriptive statistics. There are, in other words, a few methods 
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that are clearly quantitative, but the boundaries of the category are unclear. 
“Qualitative” as a descriptive default category tells us little, and in some 
cases does not even indicate whether statistics are being used. 

However, the distinction between quantitative and qualitative is often 
drawn at the level of methodology or research design (see chapter 25—Process 
Tracing). At this level, it makes even less sense than at the level of method. 
In this usage, “quantitative” is usually meant to describe a research design 
that uses statistical modeling to inferentially test formally stated hypotheses. 
Qualitative methodologies might then be understood as a default category 
covering everything else, or more narrowly as a research design that is inter- 
pretive in intent. A quantitative research design understood in this way is 
likely to include a quantitative method narrowly defined. But many research 
designs that use statistics would not be included (anything, for example, 
that uses descriptive statistics but is not designed as an exercise in inferential 
hypothesis testing). Furthermore, it is not clear where a set of comparative 
case studies designed to test hypotheses would fit, since the logic of the 
research design is inferential rather than interpretive. ' 

What people often mean when they refer to a quantitative methodology, 
in other words, is neither quantitative per se nor a methodology. It may 
be better understood as inferential or neopositivist,? and as epistemology’ 
rather than methodology. It might be contrasted most simply with an inter- 
pretive epistemology, or more robustly with a range of other understandings 
of what we are trying to do when we do social science (Jackson 2011). This 
might sound at first like a terminological quibble, but confusing our meth- 
ods with our methodologies with our epistemologies can, and often does, 
put unreasonable limits on our ability to think creatively and productively 
about using methods and about building the methodologies that will most 
effectively address our research questions. 

‘The most pernicious effect of thinking in terms of quantitative or quali- 
tative methodologies is that it often yields a set of false assumptions linking 
method, methodology, and epistemology. In the quantitative case, statistical 


1. King, Keohane, and Verba 1994 refer to such studies as qualitative but then go on 
to argue that the rules of inference that inform such studies should be the same as those in 
quantitative inferential research. 

2. I use neopositivist here following the definition of the term in Jackson 2011, with its 
focus on hypothesis testing. 

3. Epistemology referring here to general ideas about how we should go about studying 
and understanding the world, as distinct from methodology, which is a strategy to address a 
specific question. Epistemology in this sense is roughly equivalent to what Jackson 2011 refers 
to as philosophical ontology. 
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analysis is assumed to be the appropriate or preferred method for inferential 
hypothesis testing that is assumed to be the only legitimate methodology 
for epistemologically positivist social science (this is essentially the argu- 
ment that King, Keohane, and Verba 1994 are making). In the qualitative 
case, narrative case studies might be assumed to be the preferred method for 
interpretive methodologies that are assumed to be the only legitimate meth- 
odologies for epistemologically critical social science. However, neither set 
of assumptions is remotely accurate. Some methods might be more or less 
appropriate for addressing specific research questions, but there is no reason 
in principle that specific methods cannot be used within different episte- 
mologies. Thinking in terms of quantitative or qualitative methodologies 
serves only to limit our creativity in thinking about methodologies, about 
how best to answer our research questions. 

Two examples may help to illustrate this claim. Process tracing and social 
network analysis (SNA) are two methods often used in the study of interna- 
tional organizations (see chapter 25——Process Tracing and chapter 19—Social 
Network Analysis), as discussed elsewhere in this volume. They are both 
methods of organizing information; each can work well with a broad variety 
of methods for acquiring information, depending on the specific process 
being traced or social network being analyzed. They are also very differ- 
ent approaches to organizing information; process tracing looks to isolate a 
causal story within a case history while SNA looks to isolate certain kinds 
of relationships among predetermined categories of actors or institutions. 
‘The commonality between them that is key in this context is that neither is 
limited to specific epistemologies or methodologies. 

‘That they are not limited to specific methodologies has already been sug- 
gested by the observation that each can be matched to a wide variety of 
methods for gathering information. They can also be matched to a variety 
of methods for analyzing information. A network map, for example, can 
be read to determine which specific individuals or organizations are at its 
center, or can be analyzed statistically to determine its network properties. 
A process tracing narrative can be analyzed to determine primary cause for 
comparison with other cases or to understand the ways in which cause is 
complicated in the specific case. 

‘That they are not limited to specific epistemologies either is perhaps the 
more important observation in this context. Both methods, for example, 
can be used equally effectively in positivist hypothesis testing exercises and 
in interpretive analyses of specific contexts. Neither, in other words, is in 
any meaningful methodological or epistemological way either quantitative 
or qualitative per se. Efforts to put either method in either camp serve only 
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to artificially and unnecessarily delimit the utility of the method, the range 
of questions to which the method might usefully contribute to a response. 


Method, Methodology, Epistemology, and Theory 


It matters, then, whether or not we keep method and methodology conceptu- 
ally distinct from each other. All social science research is at some level mixed 
methods research, because we use different sorts of methods to do different 
sorts of things within any given research project. It does not, however, make 
sense to speak of any given research project as mixed methodology, inasmuch 
as methodology is the way we bring together methods to address a specific set 
of research questions. A research project’s methodology, understood in this 
way, is singular. But again it necessarily involves several methods, not all of 
which can be neatly categorized as quantitative or meaningfully categorized 
as qualitative. Methodology more broadly is difficult to neatly categorize, 
because it is (or at minimum should be) specific to each research project. 

Epistemology is a little trickier. When IO scholars speak of a “quantitative 
methodology” they are usually referring to a combination of a neopositivist 
epistemology, a large data set, and statistical modeling used as hypothesis 
testing. This combination makes sense in its own terms but is unnecessar- 
ily limiting. Neopositivist epistemology does not mandate either large data 
sets or statistics, and both data sets and statistical modeling can be useful in 
the context of other epistemologies. But our discipline has trained enough 
scholars in “quantitative methodology” that focuses on combining these, and 
enough other scholars have reacted against the combination, that neoposi- 
tivism, large data sets, and statistical modeling tend to be either embraced or 
rejected as a set. This may be useful for disciplinary boundary-building, but 
at the cost of needlessly constraining methodological creativity (for a more 
developed version of this argument see Barkin and Sjoberg 2017). 

When IO scholars speak of “qualitative methodology” they often mean 
quite different things by it. It may be used to refer to research that adopts 
a positivist epistemology but uses case studies, implying that epistemologi- 
cally nonpositivist work, being neither a quantitative nor qualitative meth- 
odology, is not really social science research at all (e.g., Barkin 2008). Alter- 
natively, it may refer to research that eschews positivism in favor of various 
interpretivist epistemologies. Either way there is often an assumption that 
mixing methods that do and do not use statistics is somehow conceptually 
different than mixing methods none of which use statistics. This assumption 
has a dubious epistemological basis. 
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What then of a mixed epistemology? There are different ways to look at 
this question. If one is an epistemological purist, whether that epistemology 
be neopositivist or any of a variety of antipositivist positions, then epistemol- 
ogy is prior to methodology, prior to the research question. It defines both 
what we should and what we should ot be doing as scholars of international 
organizations. Methods must still of necessity be mixed, but methodology 
must fit within the given epistemology. If one is more prone to begin with 
a research question than epistemology, then the question itself will point 
to the appropriate epistemological position. Questions about conditions of 
possibility imply different epistemological constraints, for example, then 
questions about relationships. Such scholars may not be able to successfully 
mix epistemologies within specific pieces of research but can reasonably mix 
them across questions within a broader research trajectory. 

Finally, what of theory, the thing that provides actual explanatory/inter- 
pretive/political content to our attempts to address our questions about 
international organizations?‘ Theory is in fact orthogonal not only to 
method but to methodology and epistemology as well. Take as an example 
two theoretical constructs frequently used to study international organiza- 
tions, neoliberal institutionalism (understood as the application of Coasian 
logic to IOs) and gender analysis (understood as the study of the effects of 
gender as a social or discursive construct on the practice of international 
organization). A neoliberal lens is compatible with, among other things, for- 
mal modeling (to determine what should happen in the abstract), statistical 
modeling (to determine what tends to happen), and discourse analysis (to 
determine how the discourse of neoliberalism affected regime shape in par- 
ticular historical settings). A gender lens is similarly compatible with various 
kinds of discourse analyses, but is also compatible with various kinds of 
statistical analyses, for example in answer to questions about what kinds of 
roles women play in IOs. 

In both cases, the epistemological underpinnings of the questions being 
asked about international organization and IOs differs across methodolo- 
gies. This is not to suggest that either neoliberal institutionalism or gender 
analysis are infinitely epistemologically malleable. Rather it is to suggest that 
questions of politics, of how the political world works, are distinct from 
questions of epistemology, of how we should go about studying that world. 
This is another way of phrasing a cautionary tale told often, but not often 
enough, in discussions of methods and methodology in IR generally; using 
methods and methodology well is critical to good scholarship, but we must 


4, Theory understood here in a broad and inclusive sense, to forestall argument about what 
theory really is. 
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not lose sight of the fact that our work is ultimately about the politics of 
international organizations, not methods for methods sake. 


Pluralism 


Different questions demand different mixes of methods in their method- 
ologies. Any given methodology, of course, needs to make sense in its own 
terms. It needs a supporting story of how, and in what epistemological con- 
text, the methods combine to effectively address the question. And those 
individual methods, in turn, each have their own internal rules, and can 
each be used well or poorly. Pluralism in this context is a measure of how 
willing we are to accept methodological stories that are different from the 
ones that we are used to. 

Since methodologies are the way we connect methods to research ques- 
tions, though, the more pluralist we are willing to be in our thinking about 
methodology, the more pluralist we can be in the questions that we can ask 
about international organizations. Insisting ex ante that only a limited set of 
methodological stories is acceptable as social science or as critique is in this 
sense not just an act of methodological or epistemological exclusion. It is 
also an act of political exclusion, because it limits the kinds of questions we 
can ask about politics. The key controversy in methodological pluralism in 
IO studies, therefore, is whether we are willing to allow our epistemology 
(or, worse, the limits of our methods training) to dictate what we consider 
legitimate to study. 
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69, 277-78; for document analysis, 
156; ethnographic, 35, 48-53, 269, 
276; focus groups, 80-81; language 
considerations, 78—79; LinkedIn 
research benefits, 323; observation 
and archives combined with, 277-83; 
online, 102—3; overview, 11, 67—68; 
participant observation combined 
with, 268-73; photo-elicitation, 
56-57, 135; for practice analysis, 
328-29; praxiographic, 269; for pro- 
cess tracing, 308, 312, 313; purposes 
of, 278-80; research questions for, 
68, 271, 272; with stakeholders in 
visual data, 139; tools and techniques 
for, 281-82. See also semistructured 
interviews; surveys 


keyword analysis, 287-89 
keyword searches, 127 


language considerations: focus groups, 
81; interviews, 78-79; mapmaking, 
143; negotiations, 62; participant 
observation, 36-37; surveys, 76. See 
also discourse analysis 

large-N data, quantitative analysis 
of, 223-28; benefits of, 224—25; 


challenges for, 226-28, 259; descrip- 


tion of, 223-24; methodologies vs. 


methods and, 359; for quantification 


of statistical productions, 176; struc- 
tured, focused comparison benefits, 


303; tools and techniques for, 225-26 


legal research, 116-22; challenges for, 
121-22; description of, 116-17; 
purposes of, 117-18; tools and tech- 
niques for, 118-21. See also interna- 
tional law 


legitimization of organizations: com- 
bined methods for studying, 280; 
discourse analysis of, 164; multiple 
correspondence analysis of, 240, 242— 
43; text analysis of, 232 

legitimization of researchers: challenges 
with combined methods, 280, 282-83; 
methodological choice and, 358, 361 

linear regression, in statistical analysis, 
215, 216-17, 216 

linguistic methods: for computerized 
text analysis, 287-89; observation 
combined with, 60, 62 

LinkedIn research, 323-24 


map analysis, 142-43, 144 

measurement, 11, 203-4. See also quan- 
titative methods 

measurement bias, in statistical analysis, 
217, 218 

mediation, in communication, 194 

memory, institutional, combined meth- 
ods for studying, 280, 282 

methodological pluralism, controversies 
in, 355-61 

methods, defined, 255-56 

misspecification bias, in voting analysis, 
209 

mixed epistemology, as term, 360 

mixed methods. See combined methods 

monadic format of statistical data, 215 

multilateral processes: data sources for, 
25-26; digital observation of, 31, 32; 
direct observation of, 21—26; multi- 
positionality and, 42—43; observation 
of, 59—62; practice analysis of, 325— 
31; process tracing of, 308—13; social 
network analysis of, 248-53; study of 
ideas and, 171-72 

multimethods research. See combined 
methods 

multiple correspondence analysis, 238— 
44; challenges for, 244; description 
of, 238-39; in prosopography, 320; 
purposes of, 239-40; tools and tech- 
niques for, 240-44, 243, 258 


multiplexity, in networks, 253 

multipositionality: bodily dynamic of, 
44; defined, 42; participant observa- 
tion combined with, 39, 42, 61—62; 
role of, 42—43 


national archives, 125, 126. See also 
archival methods 

negotiations: combined methods for 
studying, 282; communication by 
IOs on, 194-96; digital observation 
of, 31—32; direct observation of, 21— 
26; document analysis of, 153-56; 
history of, 120; language practices 
influence on, 62; multiplicity of 
talks in, 194; observation of, 58—62; 
participant observation of, 34-38; 
practice analysis of, 325-31; process 
tracing of, 310-13; quantitative 
methods for, 258 

neoliberal institutionalism, epistemo- 
logical considerations, 360 

neopositivist views of quantitative meth- 
odologies, 357, 359-60 

network analysis, 258-59. See also social 
network analysis 

networks, components of, 250-51 

nodes, in networks, 250, 252, 253 

nominal vs. real budgets, 190 

nondisclosure agreements, 276 

norm setting: discourse analysis of, 163— 
64; process tracing of, 310; social 
network analysis of, 250; text analysis 
of, 232, 234 

North Atlantic Treaty Organization 
(NATO): countries’ standing in, 84, 
239; expansion of, 309; feminist 
approaches to studying, 335 

note-taking: for archival research, 127, 
128; confidentiality of informants 
during, 46—47; for participant obser- 
vation, 37—38; secrecy concerns, 276; 
semistructured interviews, 85—86 

note-taking devices: for direct observation, 
25; for participant observation, 37, 38; 
for semistructured interviews, 85-86 
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observation: carnal sociology, 44—45, 
337; challenges for, 58-62, 282-83; 
defined, 19; description of, 277-78; 
digital, 31-32; diversity of approaches 
to, 58-59; exiting fieldwork, 38, 

39, 46—47; interviews and archives 
combined with, 277-83; multiposi- 
tionality in, 39, 42-43, 44, 61-62; 
overview, 11, 19—20; purposes of, 
19-20, 278-80; reversed photo- 
elicitation, 56-57; of spatial practices, 
29-30; tools and techniques for, 
281-82. See also direct observation; 
participant observation 

observational participation. See partici- 
pant observation 

online archives, 124. See also archival 
methods 

online interviews, 102-3 

ontologies, Composing Collages for, 
347-48, 349-50 

open-ended interviews. See unstructured 
interviews 

open-ended questions: semistructured 
interviews, 85; surveys, 72, 75 

ordering of survey questions, 72—73 

ordinal models, for voting analysis, 206 

Organisation for Economic Co- 
operation and Development 
(OECD): multiple correspondence 
analysis of, 240, 241, 242-44, 243; 
OECD-DAC databases, 188 

organizational dynamics: discourse 
analysis of, 164; document analysis 
of, 114, 153; process tracing of, 309; 
quantification studies on, 176; secrecy 
concerns, 275; social network analysis 
of, 250; structured, focused compari- 
son of, 303; tweet analysis of, 160, 
161; visual methods for, 135 


panel data, defined, 223 

participant observation, 33—39; access 
to organizations, 36, 38, 61-62; 
challenges for, 38-39, 272-73; for 
challenging secrecy, 276; description 
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participant observation (continued) 
of, 33—34, 268-69; direct observation 
vs., 21; exiting of, 46—47; interviews 
combined with, 268-73; multiposi- 
tionality combined with, 39, 42; for 
practice analysis, 328, 329; for process 
tracing, 308; purposes of, 34-35; for 
social network analysis, 251; tools and 
techniques for, 35-38, 37, 271-72 

participatory visual methods, 135, 139 

passivization, awareness of, 344 

patterns of action, 325. See also practice 
analysis 

photo-elicitation: defined, 135; reversed, 
56-57 

photography, of archival documents, 
127-28. See also visual methods 

photo-voice, defined, 135 

piracy reduction campaign, document 
analysis of, 154-55, 156 

pivot tables, in prosopography, 319 

political processes: budget setting, 187, 
188, 190; language considerations, 
62; quantification of statistical pro- 
ductions, 179; recursive relationship 
with legal processes, 118, 120-21; 
statistical analysis of IO data, 218; 
text analysis of, 230, 231; visual 
methods for, 139 

positionality of researchers: in Com- 
posing Collages, 350; disclosure of, 
198; in feminist approaches, 337-38; 
validity of interviewing and, 106-7. 
See also multipositionality 

positional practices, 327. See also prac- 
tice analysis 

positivist tradition: in combined meth- 
ods, 358, 359; for interviews, 105, 
106; in legal research, 117, 119-20 

postcolonial insights, 129, 130, 338, 
341-42 

power relations: asymmetrical interviews 
and, 86-87, 92-93, 108-9, 312; 
in beneficiary interviews, 90; carnal 
sociology influence on, 44—45; in 
communication, 195; discourse analy- 


sis of, 164; feminist approaches to 
studying, 333-34, 335-37, 338; dur- 
ing interviews, 104, 108-9; LinkedIn 
research for studying, 324; in maps 
and mapmaking, 143; participant 
observation of, 34; process tracing of, 
310; in semistructured interviews, 83, 
86-87; spatial dimensions of, 29. See 
also hierarchical relationships 

practice analysis, 325-31; challenges for, 
329-31; description of, 325—26; in 
feminist approaches, 337; purposes 
of, 326-27; tools and techniques for, 
327-29 

practices: defined, 325; direct observa- 
tion of, 22, 23 

practice theorizing, 152 

praxiographic interviews, 269 

praxiography: challenges for, 155—56; 
description of, 152; purposes of, 152; 
tools and techniques for, 153-55 

preprocessing, for text analysis, 232 

presentation of results: Composing Col- 
lages, 348; process tracing, 313; qualita- 
tive comparative analysis, 297; Reflexive 
Discourse Analysis of, 34344; struc- 
tured, focused comparison, 306 

privacy: for biographic interviews, 97— 
98, 99-100; data protection concerns, 
25; ethical issues, 12; intrusion of 
private life in digital public meetings, 
32. See also confidentiality concerns 

private collections, 125, 126. See also 
archival methods 

process dynamics, as term, 310 

process tracing, 308-13; budget 
influencing factors, 188; challenges 
for, 313; description of, 309; of idea 
development, 172; in participant 
observation, 35; purposes of, 309— 
11, 358—59; in structured, focused 
comparison, 302, 306; through 
document analysis, 153-55; tools and 
techniques for, 311-12 

Professionals in Humanitarian Assistance 
and Protection, 74 


prosopography, 315-21; challenges for, 
320-21; description of, 315-16; mul- 
tiple correspondence analysis of, 239; 
for process tracing, 312; purposes of, 
316-17; tools and techniques for, 
317-20, 318 

purposive sampling, 84 

p-value threshold, in linear regression, 217 


qualitative comparative analysis, 292— 
99; challenges for, 298-99; descrip- 
tion of, 292-93; purposes of, 293-94; 
tools and techniques for, 295-97, 296 

qualitative methodology, as term, 359 

qualitative methods: methodological 
divide with quantitative methods, 
302-3; methodologies vs. methods, 
356-59; quantitative methods inte- 
grated with, 204, 223, 226. See also 
combined methods; specific methods 

quality of data. See data quality 

quantification of statistical productions, 
174-80; challenges for, 179-80; 
description of, 174-75; purposes of, 
176; tools and techniques for, 177— 
79, 184 

quantitative methodology, as term, 359 

quantitative methods: databases on 
individuals, 246-47, 258; increasing 
importance of, 256, 260; large-N data 
quantitative analysis, 176, 223-28, 
259, 303, 359; methodological divide 
with qualitative methods, 302-3; 
methodologies vs. methods, 356-59; 
multiple correspondence analysis, 
238-44, 243, 258, 320; qualitative 
methods integrated with, 204, 223, 
226; shortcomings in current use 
of, 255-60, 333-34; social network 
analysis, 248-53, 252, 258-59, 320, 
324, 358-59; voting analysis, 205-9, 
227, 256, 257. See also combined 
methods; computerized text analysis; 
statistical analysis 

questionnaires, for prosopography, 317, 
318-19, 318 
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guides; research questions; survey 
questions 

quota sampling, 73 


R (statistical software program), 241, 
244, 297 

real vs. nominal budgets, 190 

recall bias, 72 

Reflexive Discourse Analysis (RDA), 
343-44 

reflexivity: in carnal sociology, 44, 45; 
in communication by IOs, 193-98; 
in Composing Collages, 350; in 
data collection and generation, 25; 
in digital observation, 32; in dis- 
course analysis, 168; in ethnographic 
interviews, 48, 50; in focus groups, 
81; importance of, 4; in interviewing, 
68, 90, 107, 109; longitudinal study 
results, 7, 8, 9-10, 70; in multipo- 
sitionality, 43; in participant obser- 
vation, 39; practice of, 343—44; in 
reversed photo-elicitation, 56—57 

refugee flows, statistical analysis of, 212— 
18, 214, 216 

regression models: for limited dependent 
variables, 226; linear regression, 215, 
216-17, 216 

relational practices, 327. See also practice 
analysis 

relational vs. afhliation networks, 251 

Reliefweb, 212 

research questions: artifact analysis, 149; 
branding analysis, 146-47; budget 
analysis, 186; Composing Collages, 
346-47, 349; computerized text 
analysis and archival methods, 285— 
86, 287; databases on individuals, 
247; dataviz analysis, 183, 184; direct 
observation, 22, 23—24; document 
analysis, 153, 154-55; documenting, 
114-15; epistemological consider- 
ations, 360-61; ethnographic inter- 
views, 50—51; expedition effects on 
on, 353—54; interviews, 
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research questions (continued) 
68; interviews and observation, 271, 
272; legal research, 117, 118-21; 
measuring, 203—4; methodologies vs. 
methods and, 356; multiple cor- 
respondence analysis, 240, 243-44; 
observation, 19—20; parallels between 
IO studies and broader disciplines, 
259; participant observation, 37-38; 
qualitative comparative analysis, 292; 
quantification of statistical produc- 
tions, 174, 177, 178, 179; semiol- 
ogy of websites, 158; semistructured 
interviews, 83—84; statistical analysis, 
225; structured, focused comparison, 
301, 304, 305, 306; text analysis, 
231-32, 235 

revenues, defined, 187. See also budget 
analysis 

reversed photo-elicitation, 56—57 

Rio+20 Conference (2012), 59-60 

robustness testing: qualitative compara- 
tive analysis, 297; statistical analysis, 
226 

rules: direct observation of, 23; spatial 
dimensions of, 29; text analysis of, 
232 


sample network graphs, 252, 252 

sampling techniques: qualitative com- 
parative analysis, 294, 298; semis- 
tructured interviews, 84-85; surveys, 
73-74, 75 

secrecy, challenging, 275-76, 280 

self-presentation by biographic inter- 
viewers, 97 

self-presentation by organizations: 
combined methods for studying, 
279-80, 281-82; limitations of, 268; 
stage management for, 270-71, 273; 
through documents, 114; through 
interviews, 68; through texts, 231; 
visual methods for, 135 

self-resocialization, in Reflexive Dis- 
course Analysis, 344 

semiology of websites, 158-59 


semistructured interviews, 82—87; chal- 
lenges for, 83, 86-87; for challenging 
secrecy, 276; defined, 82; descrip- 
tion of, 82-83, 269; ethnographic 
interviews combined with, 51; for 
process tracing, 312; purposes of, 83— 
84; tools and techniques for, 84-86; 
validity of, 105 

sentiment analysis, 234. See also com- 
puterized text analysis 

“Set Speech” obstacles, 81, 91 

set theory, in qualitative comparative 
analysis, 293 

settings: for biographic interviews, 
97-98; direct observation of, 22, 23, 
24; factors in choice of, 24; feminist 
approaches to studying, 337-38; for 
interviewing beneficiaries, 90; for 
online interviews, 103; postcolonial 
approaches to studying, 341 

shared preferences, voting analysis of, 
207-9 

single-case studies, limitations of, 303 

sites. See settings 

situational practices, 327. See also prac- 
tice analysis 

smoking-gun observations, in process 
tracing, 311 

snowball sampling, 50, 84-85 

social desirability bias, 72 

social life of ideas, 171-72 

social media: communication of IOs via, 
198; LinkedIn research, 323—24; for 
survey sampling, 74; tweet analysis, 
160-61 

social network analysis, 248—53; 
challenges for, 253; description of, 
248-49; LinkedIn research benefits, 
324; in prosopography, 320; purposes 
of, 249-50, 358-59; as quantitative 
method, 258-59; tools and tech- 
niques for, 250-52, 252 

sociographic approaches, 315. See also 
prosopography 

software: for computerized text analysis, 
288; for content analysis, 136; for 


managing archival research, 128; for 
multiple correspondence analysis, 
241, 244; for qualitative comparative 
analysis, 297; for voting analysis, 206 

Somalia, reduction of piracy, 154-55, 
156 

spatial models, for voting analysis, 206 

spatial practices observation, 29-30 

speaking time, traffic lights for, 149-50, 
150 

staff of international organizations. See 
workforce of international organiza- 
tions 

stage management by international 
organizations, 270-71, 273 

standard error, in linear regression, 217 

Stata (software), 297 

statistical analysis, 212-18; challenges 
for, 217-18, 259; description of, 
212-13; human resource statistics, 
221-22; methodologies vs. methods 
and, 356-58, 359; in official com- 
munications of IOs, 196; in proso- 
pography, 319, 320-21; purposes 
of, 213-15, 214; quantification of 
statistical productions, 174-80; tools 
and techniques for, 215-17, 216 

stereotypes: in statistical productions, 
180; in visual products, 139 

storylines, in process tracing, 311, 312 

strategic misspecification bias, in voting 
analysis, 209 

structured, focused comparison, 301-5 

structured interviews: description of, 
269; semistructured interviews vs., 
82. See also interviews 

studying ideas, 171-72 

subject positioning, awareness of, 344 

summary statistics, 215. See also statisti- 
cal analysis 

survey questions: definitive nature of, 
76; design of, 72-73; for prosopogra- 
phy, 318, 318, 319 

surveys, 69-76; challenges for, 74—75; 
for data collection, 228; defined, 
69; description of, 69-70; design of, 
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71-75, 74; for prosopography, 317, 

318-19, 318; purposes of, 70-71; for 

social network analysis, 251-52 
symmetric vs. asymmetric networks, 251 


tagging, for computerized text analysis, 
288 

text analysis: description of, 152; for 
postcolonial investigations, 341; 
for practice analysis, 328, 329; as 
quantitative method, 258-59. See also 
computerized text analysis; document 
analysis 

thematic maps, 142 

theoretical frameworks: for discourse 
analysis, 168; interpretation of results 
influenced by, 107; orthogonality to 
methods, methodology, and epis- 
temology, 360-61; semistructured 
interviews, 84 

thick descriptions: in observation, 19; in 
participant observation, 33, 38, 39; in 
spatial practices observation, 29-30 

ties, in networks, 250, 253 

time-series datasets, defined, 223 

topic modeling, for text analysis, 233 

Tosmana (software), 297 

traffic lights (meeting devices), 149-50, 
150 

transcriptions: ethnographic interviews, 
53; participant observation, 37; semi- 
structured interviews, 85—86. See also 
note-taking 

translation: during focus groups, 81; 
during interviews, 79; of surveys, 76; 
text analysis challenges, 234, 235, 259 

travel: archive visits, 127-29, 133, 137; 
for document analysis, 155-56 

treaties: legal underpinnings of, 116, 
117; negotiating history of, 120 

triangulation: for biographic interviews, 
99; combined methods for, 266; for 
ethnographic interviews, 52; for inter- 
views, 106; for process tracing, 308, 
312, 313; semistructured interviews 
for, 82, 83, 84, 87 
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truth tables, in qualitative comparative 
analysis, 295, 297 

T value, in linear regression, 217 

Twitter, analysis of tweets from, 160-61 


typologies of international organizations: 


overview, 3; qualitative comparative 
analysis of, 294 


UCDP Armed Conflict Dataset (v 19.1), 
216 

uncertainty, in statistical analysis, 213, 
215, 217 

unclosing international organizations, 
through biographic interviews, 95, 
96-97 

UNESCO: budget setting, 187; climate 
change initiatives, 196; semiology of 
website of, 159; statistical analysis 
reform, 176 

United Nations: Agenda for Sustainable 
Development, 308, 311; archives of, 
127, 129, 132; branding analysis of, 
146-47, 147; budget analysis of, 191; 
countries’ standing in, 84, 239; data- 
base on peacekeeping leaders, 246— 
47; demographic data, 177; General 
Assembly voting, 205, 206, 207, 208, 
209, 257; Geospatial Information 
Section, 143; groups of actors within, 
22; human resource data, 221-22; 
influence of setting on behavior, 
29-30; participant observation in, 
34; postwar development projects, 
195—96; Twittersphere of, 160-61; 
women's rights work, 336; working 
languages of, 76 

United Nations Charter, 120 

United Nations Committee for Food 
Security, 61—62 

United Nations Development Pro- 
gramme, 269, 334 

United Nations Environment Pro- 
gramme, 269 

United Nations Food and Agricultural 
Organization: agricultural biotech- 
nology discourse, 22; biotechnology 


report production, 59; combined 
methods for studying, 278; Food 
Security program, 60, 61-62 

United Nations High Commissioner for 
Refugees (UNHCR): budget analysis 
of, 191; ILO memorandum of under- 
standing, 288; legal research on, 118; 
statistical analysis of data produced 
by, 212-18, 214, 216; use of images, 
136, 137, 138 

United Nations History Project, 129 

United Nations Human Rights Council, 
257 

United Nations Office for the Coor- 
dination of Humanitarian Affairs 
(OCHA), 212 

United Nations Refugee Agency, 80-81 

United Nations Secretariat, 221 

United Nations Security Council: femi- 
nist approaches to studying, 334, 335; 
legal research on, 118; voting analysis 
of, 206, 208-9; Women and Peace 
and Security resolution, 341 

United Nations System Chief Executives 
Board (UNSCEB), 188, 221-22 

United Nations Women, 42—43 

unpublished documents, in archives, 
124-26 

unstructured interviews: description of, 


269; semistructured interviews vs., 
82. See also interviews 


validity of research: budget analysis, 
189, 191; computerized text analy- 
sis and archival methods, 287, 290; 
interviews, 104-8, 109; qualitative 
comparative analysis, 297; structured, 
focused comparison, 302-3, 306 

vdem dataset (Coppedge et al.), 216 

virtual interviews, 102—3 

virtual meetings, digital observation of, 
31-32 

visual archives, 132-33 

visualization, in prosopography, 320 

visual methods, 134—40; challenges for, 
139-40; description of, 134-35; for 


official communications of IOs, 196— 
97; purposes of, 135-36; tools and 
techniques for, 136-39. See also data- 
viz analysis; map analysis; reversed 
photo-elicitation 

vote buying, 208-9 

voting analysis, 205-9, 227, 256, 257 


wayback machine, for websites, 159 

web-based surveys, 71, 73 

websites, semiology of, 158-59 

whole vs. egocentric networks, 251 

women’s rights work, 336. See also femi- 
nist approaches 


Wordfish, 233-34 


workforce of international organizations: 
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biographic interviews of, 94-100; 
databases on individuals, 246-47; 
dataviz production by, 182-83; 
human resource statistics, 221—22; 
participant observation by, 36; surveys 
for studying, 70-71 

World Bank, 334 

World Conservation Congress, 22 

World Health Organization (WHO), 
136, 149-50, 150 

World Intellectual Property Organiza- 
tion (WIPO), 61 

World Values Survey (WVS), 72 


Yearbook of International Organiza- 
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